devlyn-cli 2.10.1 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +4 -9
- package/CLAUDE.md +11 -15
- package/README.md +31 -7
- package/benchmark/auto-resolve/results/20260507-f10-f11-tier1-full-pipeline/headroom-gate.json +25 -0
- package/benchmark/auto-resolve/results/20260507-f10-f11-tier1-full-pipeline/headroom-gate.md +10 -0
- package/benchmark/auto-resolve/results/20260508-f22-exact-error-headroom/headroom-gate.json +18 -0
- package/benchmark/auto-resolve/results/20260508-f22-exact-error-headroom/headroom-gate.md +9 -0
- package/benchmark/auto-resolve/results/20260508-f26-headroom/headroom-gate.json +18 -0
- package/benchmark/auto-resolve/results/20260508-f26-headroom/headroom-gate.md +9 -0
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F16-cli-quote-tax-rules/l2_risk_probes/result.json +28 -0
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F23-cli-fulfillment-wave/l2_risk_probes/result.json +28 -0
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F25-cli-cart-promotion-rules/l2_risk_probes/result.json +30 -0
- package/benchmark/auto-resolve/results/20260511-f12-webhook-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260511-f12-webhook-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260511-f15-concurrency-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260511-f15-concurrency-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/F21-cli-scheduler-priority/l2_risk_probes/result.json +32 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/full-pipeline-pair-gate.json +44 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/full-pipeline-pair-gate.md +16 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/headroom-gate.json +18 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/headroom-gate.md +9 -0
- package/benchmark/auto-resolve/results/20260511-f3-http-error-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260511-f3-http-error-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f2-medium-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f2-medium-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f31-seat-rebalance-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f31-seat-rebalance-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f4-web-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f4-web-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f5-fixloop-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f5-fixloop-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f6-checksum-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f6-checksum-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f7-scope-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f7-scope-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f9-e2e-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f9-e2e-headroom/headroom-gate.md +15 -0
- package/bin/devlyn.js +6 -1
- package/config/skills/_shared/.ruff.toml +2 -0
- package/config/skills/_shared/adapters/claude.md +9 -1
- package/config/skills/_shared/adapters/codex.md +5 -0
- package/config/skills/_shared/adapters/grok.md +4 -2
- package/config/skills/_shared/archive_run.py +738 -56
- package/config/skills/_shared/codex-config.md +19 -4
- package/config/skills/_shared/codex-monitored.sh +81 -12
- package/config/skills/_shared/collect-codex-findings.py +311 -128
- package/config/skills/_shared/engine-preflight.md +8 -7
- package/config/skills/_shared/expected.schema.json +61 -0
- package/config/skills/_shared/finish-gate.py +12 -11
- package/config/skills/_shared/grok-anchor-guard.py +18 -1
- package/config/skills/_shared/invocation-receipt.py +1041 -0
- package/config/skills/_shared/judge-output-parser.py +200 -0
- package/config/skills/_shared/judge-role-evidence.py +273 -0
- package/config/skills/_shared/process-evidence.py +909 -0
- package/config/skills/_shared/resolve-bootstrap.py +776 -84
- package/config/skills/_shared/resolve-stop-hook.py +33 -1
- package/config/skills/_shared/role-config.py +552 -0
- package/config/skills/_shared/run-bounded.py +1 -1
- package/config/skills/_shared/spec-verify-check.py +749 -181
- package/config/skills/_shared/state-phase-write.py +1838 -58
- package/config/skills/_shared/terminal-claim-check.py +15 -0
- package/config/skills/_shared/verify-merge-findings.py +958 -51
- package/config/skills/devlyn:engines/SKILL.md +8 -16
- package/config/skills/devlyn:ideate/references/spec-template.md +1 -0
- package/config/skills/devlyn:queue/SKILL.md +7 -4
- package/config/skills/devlyn:resolve/SKILL.md +91 -33
- package/config/skills/devlyn:resolve/references/outer-loop.md +15 -0
- package/config/skills/devlyn:resolve/references/phases/build-gate.md +31 -2
- package/config/skills/devlyn:resolve/references/phases/implement.md +3 -2
- package/config/skills/devlyn:resolve/references/phases/verify.md +120 -84
- package/config/skills/devlyn:resolve/references/state-schema.md +22 -8
- package/package.json +5 -1
- package/scripts/lint-skills.sh +314 -285
|
@@ -24,24 +24,16 @@ DEVLYN_SHARED_DIR="$(cd "$DEVLYN_SKILL_DIR/../_shared" && pwd)"
|
|
|
24
24
|
|
|
25
25
|
## No args — status + how to choose
|
|
26
26
|
|
|
27
|
-
1.
|
|
28
|
-
2. Run `bash "$DEVLYN_SHARED_DIR/engine-doctor.sh"`
|
|
29
|
-
3.
|
|
30
|
-
|
|
31
|
-
```
|
|
32
|
-
Role Resolved Source Available engines
|
|
33
|
-
orchestrator (this CLI) not configurable — switch CLIs to change
|
|
34
|
-
executor claude default claude ✓ codex ✓
|
|
35
|
-
pair judge codex binary rule (pin order with: pair <name>,...)
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
4. Show the script's full detection table plus its diversity/recommendation line beneath the role table.
|
|
39
|
-
5. End with one usage line per subcommand below.
|
|
27
|
+
1. Run `python3 "$DEVLYN_SHARED_DIR/role-config.py" --workdir "$PWD" --default-engine <orchestrator-supported-default>` and print each resolved role's engine, requested model/effort, source and channel. Null model/effort means inherited/unresolved, not an attested identity. Show unavailable pins and requested priority even when no OTHER is available; status does not authorize dispatch. A current run can instead be inspected with `--state .devlyn/pipeline.state.json`; do not reread config as though it changed the active run. Show returned observations with their evidence basis; effective effort remains unknown when only argv is bound.
|
|
28
|
+
2. Run `bash "$DEVLYN_SHARED_DIR/engine-doctor.sh"` and show its availability/eligibility table. This does not prove authentication or model fitness.
|
|
29
|
+
3. Show the subcommands below. Status never launches a model.
|
|
40
30
|
|
|
41
31
|
## Subcommands
|
|
42
32
|
|
|
43
|
-
- `executor <name>` — pin the executor
|
|
33
|
+
- `executor <name>` — pin the legacy executor; optional worker/primary profiles take their documented precedence. PLAN is orchestrator-fixed and never follows the pin. Refuse names with no `_shared/adapters/<name>.md`, or whose adapter's `## Role eligibility` section declares `executor: no` (judge-only backends), listing the valid names either way. If the engine's CLI is not currently available, still write the pin but warn: pins are promises — the run will stop with `BLOCKED:<name>-unavailable` until it is installed.
|
|
44
34
|
- `pair <name>[,<name>...]` — set `pair_judge_priority` (ordered; first adapter-valid, pair-judge-eligible, non-primary, available entry wins at VERIFY/risk-probe time). Refuse names with no `_shared/adapters/<name>.md`, or whose adapter declares `pair_judge: no`.
|
|
45
|
-
- `
|
|
35
|
+
- `role <worker|primary_judge|pair_judge> <JSON object>` — write one atomic role entry, e.g. `{"engine":"codex","model":"gpt-6-astra","effort":"high"}`. Invoke `python3 "$DEVLYN_SHARED_DIR/role-config.py" --workdir "$PWD" --default-engine <orchestrator-supported-default> --set-role <role> --value '<JSON object>'` using structured argv, never shell interpolation of user text. Unknown fields/types and ineligible roles fail without changing prior bytes.
|
|
36
|
+
- `role <name> clear` — same command with `--value clear` removes only that override.
|
|
37
|
+
- `clear` — invoke the helper with `--set-role clear` to remove legacy and role pins; preserve unrelated keys and delete an empty file.
|
|
46
38
|
|
|
47
|
-
Writes preserve
|
|
39
|
+
Writes preserve unrelated top-level keys already in the file; unknown fields inside roles are errors. After every change, re-print the resolved role table. Never modify anything outside `.devlyn/engines.json`, and never launch a pipeline from this skill.
|
|
@@ -115,6 +115,7 @@ Substantive (ideate's job during elicitation):
|
|
|
115
115
|
- Each Constraint has a reasoning clause.
|
|
116
116
|
- Out of Scope explicitly enumerates non-goals.
|
|
117
117
|
- Verification commands actually verify the Requirement they map to (no "looks plausible" verification).
|
|
118
|
+
- Avoid formatter-sensitive single-line adjacency checks — prefer a semantic test (runtime, DB, or CLI check), or grep one stable line-local anchor and assert required context separately.
|
|
118
119
|
- Frontmatter `complexity` reflects verification shape: `high` when the spec combines state mutation with ordering/priority, idempotency, auth/error priority, rollback/failure handling, or exact output shape; `medium` for ordinary multi-step work; `trivial` only for a single localized behavior.
|
|
119
120
|
- When Requirements combine state mutation, ordering/priority, idempotency, auth/error priority, or exact output shape, Verification includes at least one compound scenario that exercises the interaction end-to-end; isolated happy paths are insufficient.
|
|
120
121
|
- For benchmark, risk-probe, or pair-evidence specs, include a solo-headroom hypothesis inside `## Verification`: the artifact must literally contain `solo-headroom hypothesis`, `solo_claude`, `miss`, and a backticked observable command while naming the visible behavior a capable `solo_claude` baseline is expected to miss; the backticked line itself must contain `miss`, be framed as the command/observable that exposes it, and match a `spec.expected.json.verification_commands[].cmd` entry. For regenerated pair evidence, this hypothesis is the source for VERIFY's canonical `spec.solo_headroom_hypothesis` trigger reason and must pass `benchmark audit --require-hypothesis-trigger`. If the hypothesis is only "the task is hard", rework the spec before measurement.
|
|
@@ -20,6 +20,9 @@ fi
|
|
|
20
20
|
DEVLYN_SHARED_DIR="$(cd "$DEVLYN_SKILL_DIR/../_shared" && pwd)"
|
|
21
21
|
```
|
|
22
22
|
|
|
23
|
+
Before processing the first item, read and obey
|
|
24
|
+
`../devlyn:resolve/references/outer-loop.md`. Its scoped-commit order is binding.
|
|
25
|
+
|
|
23
26
|
## No args — status
|
|
24
27
|
|
|
25
28
|
Read `docs/specs/queue.md` (absent → report "queue empty — nothing staged" and how to add). Print pending `[ ]`, done `[x]`, and blocked `[F]` counts, the next item up, and one usage line per subcommand.
|
|
@@ -28,11 +31,11 @@ Read `docs/specs/queue.md` (absent → report "queue empty — nothing staged" a
|
|
|
28
31
|
|
|
29
32
|
- `add <intent text>` — append `- [ ] <intent>` to `docs/specs/queue.md` (create the file with its header if missing). If the intent came out of a conversation that already produced a spec, link it: `- [ ] (spec: docs/specs/<id>/spec.md) <intent>`.
|
|
30
33
|
- `drain` — serial drain per the project-instructions contract. For each pending item, in order:
|
|
31
|
-
1. Spec it if unspecced (the queue entry is the user's go-ahead). Unattended assumptions may only take scope-narrowing, reversible, non-user-visible defaults; material ambiguity (user-visible behavior, data/state semantics, new files/scripts/flags, implementation surface) → mark `[F] needs-review: <question
|
|
32
|
-
2.
|
|
34
|
+
1. Spec it if unspecced (the queue entry is the user's go-ahead). Unattended assumptions may only take scope-narrowing, reversible, non-user-visible defaults; material ambiguity (user-visible behavior, data/state semantics, new files/scripts/flags, implementation surface) → mark `[F] needs-review: <question>`, commit that queue transition, and continue.
|
|
35
|
+
2. Commit the current queue-item delta and linked spec bundle as the scoped owner baseline, then run `/devlyn:resolve --spec <path>` hands-free.
|
|
33
36
|
After every resolve invocation, run `python3 "$DEVLYN_SHARED_DIR/terminal-claim-check.py" .`; exit 79 marks `[F] FAILED-INCOMPLETE` from the predicate, never from the session self-report.
|
|
34
|
-
3. Outer loop on the terminal verdict: PASS → mark `[x]`. Findings-backed verdicts (NEEDS_WORK, verify/build-gate exhaustion) → amend the spec
|
|
35
|
-
4. A blocked item never halts the queue
|
|
37
|
+
3. Outer loop on the terminal verdict: PASS → mark `[x]`. Findings-backed verdicts (NEEDS_WORK, verify/build-gate exhaustion) → amend the spec, commit that scoped amendment, then re-run — at most 3 outer iterations. Infrastructure / invalid-input / engine-availability / implement-empty BLOCKED verdicts are not spec-amendable → mark `[F] <verdict>` immediately.
|
|
38
|
+
4. Commit each terminal `[x]` / `[F]` queue transition before advancing. A blocked item never halts the queue.
|
|
36
39
|
5. When the queue is drained (or the session must stop), emit the drain report: per-item verdict, every logged assumption, and the commit range produced.
|
|
37
40
|
|
|
38
41
|
Strictly SERIAL — one `/devlyn:resolve` run at a time, never parallel. Never invent queue items, never reorder them, and never delete an item — only mark `[x]` / `[F]`.
|
|
@@ -54,7 +54,7 @@ Each phase routes to an engine and prepends the per-engine adapter header from `
|
|
|
54
54
|
- Claude Code phases other than PLAN: spawn `Agent` (`mode: "bypassPermissions"`); prompt = adapter-header + canonical-body + task-context. PLAN uses the native foreground call shape in PHASE 1.
|
|
55
55
|
- Codex CLI phases: shell out via `bash "$CODEX_MONITORED_PATH"` with the same compounded prompt. Each `codex exec` child is a new session/fresh context. The wrapper closes stdin and emits a heartbeat. No MCP. The wrapper call is foreground-blocking — never a background shell (`run_in_background`, `&`, `nohup`), never end your message while it runs: headless print-mode wind-down kills backgrounded children (0-byte delivery); the heartbeat is the observability channel.
|
|
56
56
|
- oh-my-pi phases: spawn the native `task` tool with a fresh `context` containing adapter-header + canonical-body + task-context. Capture the task result into `.devlyn/<phase>.stdout` and any tool error into `.devlyn/<phase>.stderr` before updating state. If the `task` tool is unavailable for an omp-routed phase, write the current phase verdict as `"BLOCKED"` and report `BLOCKED:fresh-context-unavailable`; do not fall back to same-context execution or a nested `omp -p` subprocess.
|
|
57
|
-
- Default engine: Claude
|
|
57
|
+
- Default engine: Claude when the orchestrator has Claude Code’s native `Agent`; otherwise its own fresh worker. PLAN is orchestrator-fixed and never inherits `--engine`, an executor pin, or `state.engine`. BUILD_GATE and probes retain their existing routes. `_shared/engine-preflight.md#role-resolution` defines the single resolver and explicit `--role-config` / project role precedence. Absent profiles preserve legacy `--engine` / executor / default behavior; VERIFY primary may be independently selected. Configured role/priority pins and flags fail closed on unavailable engine or failed authentication, including failures discovered at dispatch. Unconfigured automatic VERIFY selects an available OTHER engine.
|
|
58
58
|
- The `--engine` flag does not disable default pairing: the second judge uses the OTHER engine by default when available.
|
|
59
59
|
- Multi-LLM evolution: when a new model adapter ships in `_shared/adapters/`, that engine becomes selectable for the roles its adapter declares eligible without further skill changes (NORTH-STAR.md "Multi-LLM evolution direction").
|
|
60
60
|
</engine_routing>
|
|
@@ -83,33 +83,43 @@ For every direct complete→spawn handoff, call `state-phase-write.py ... --phas
|
|
|
83
83
|
DEVLYN_DEFAULT_ENGINE="<current-cli-default>" python3 "$DEVLYN_SHARED_DIR/resolve-bootstrap.py" <pipeline_config tokens>
|
|
84
84
|
```
|
|
85
85
|
|
|
86
|
-
Read its sole JSON result. On `ok:false`, halt on its exact report-level `blocked` string and show `detail`; init failures
|
|
86
|
+
Read its sole JSON result. On `ok:false`, halt on its exact report-level `blocked` string and show `detail`; init failures create no phase verdict; crashes can leave owned residue, which bootstrap preserves and refuses to replace. Admission refusals require continuing the existing run through its owning session or starting in a distinct worktree. Never autoarchive to defeat a refusal; explicit manual archive is recovery only after the operator establishes that prior writers stopped. Valid completed runs still autoarchive normally, but a completed final report does not prove every interactive writer exited. On success, the script has atomically initialized the schema-v3 skeleton (`pair_verify: true` only when `--pair-verify` was passed), stamped the null-safe Claude session id, persisted exact-byte Goal/spec identity, staged spec verification inputs, and captured the verify-only external diff. It validates only flags needed for those init fields, including mode exclusivity, `--max-rounds`, `--bypass`, and `--pair-verify`/`--no-pair`; `--pair-verify` and `--no-pair` are mutually exclusive. Free-form init sets `state.source.type = "generated"`. Spec staging validates supported `complexity` frontmatter (including sibling spec `complexity` frontmatter) and any present actionable solo-headroom hypothesis, matching its command to `spec.expected.json.verification_commands[].cmd` or the inline `## Verification` JSON carrier. It stages and validates an explicit `--role-config` object into state with its path/digest, but does not resolve roles, write the untracked baseline, classify complexity/risk or announce. `state.engine` is the raw `--engine` value with `engine_source: "flag"`, otherwise the passed `DEVLYN_DEFAULT_ENGINE` with `engine_source: "default"`; step 2 resolves and replaces both fields.
|
|
87
87
|
|
|
88
|
-
2. Engine pre-flight: follow `_shared/engine-preflight.md
|
|
88
|
+
2. Engine pre-flight: follow `_shared/engine-preflight.md`. Freeze the shared resolver once before any phase with `python3 "$DEVLYN_SHARED_DIR/state-phase-write.py" --devlyn-dir .devlyn --freeze-roles --default-engine "<current-cli-default>"`. It validates project/per-run roles and records `state.role_resolution`; legacy `state.engine`/`engine_source` remain the executor. Repeated reads return the same snapshot, never changed project settings. Keep availability/auth checks before each selected dispatch; explicit unavailable routes fail closed. These PHASE0 failures are report-level `BLOCKED:<reason>`; phase verdict carriers remain bare enums.
|
|
89
89
|
|
|
90
90
|
3. Write `.devlyn/untracked.baseline`: `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --write-untracked-baseline`.
|
|
91
91
|
|
|
92
92
|
4. Read the Goal/spec through `state.source`. For free-form mode, run the deterministic classifier in `references/free-form-mode.md`. Zero-scope-signal goals halt with `BLOCKED:large-needs-ideation`; pair-evidence intent without an actionable solo-headroom hypothesis must halt with `BLOCKED:solo-headroom-hypothesis-required`; unmeasured pair-candidate intent without solo ceiling avoidance must halt with `BLOCKED:solo-ceiling-avoidance-required`. Follow the selected branch to write the trivial/medium/large body of `.devlyn/criteria.generated.md`, then set `state.complexity` and its raw-byte `criteria_sha256`. The Large `## Assumptions`/recommendation/final-report obligations remain unchanged.
|
|
93
93
|
|
|
94
|
-
Compute `state.risk_profile` from the user goal plus spec/criteria text. Mark `high_risk: true` for auth/authz, permissions, security, token/session, payment/money/billing/invoice/pricing/tax/ledger, persistence/data mutation/deletion/migration, idempotency/replay/duplicate, API/webhook/raw-body/signature, allocation/scheduling/inventory/rollback/transaction, or explicit error-priority/output-shape contracts. Explicit `--risk-probes` sets both probe booleans true. Otherwise an automatic high-risk route enables probes only when the
|
|
94
|
+
Compute `state.risk_profile` from the user goal plus spec/criteria text. Mark `high_risk: true` for auth/authz, permissions, security, token/session, payment/money/billing/invoice/pricing/tax/ledger, persistence/data mutation/deletion/migration, idempotency/replay/duplicate, API/webhook/raw-body/signature, allocation/scheduling/inventory/rollback/transaction, or explicit error-priority/output-shape contracts. Explicit `--risk-probes` sets both probe booleans true. Otherwise an automatic high-risk route enables probes only when the legacy executor’s OTHER engine (legacy pair priority/complement, independent of VERIFY profiles) is available and `--no-risk-probes` is absent; if unavailable, keep probes disabled and append `auto-risk-probes skipped: <engine>-unavailable`. `--no-pair` sets `pair_default_enabled: false`. Preserve strict boolean/list types and concise string reasons.
|
|
95
95
|
|
|
96
96
|
5. Announce one line: `resolve starting — run <run_id> — engine <engine> — mode <mode> — complexity <complexity-or-na> — pair <on|solo:auto_pair_other_engine_unavailable|disabled> — risk_probes <on|off>`.
|
|
97
97
|
|
|
98
|
-
6.
|
|
98
|
+
6. Open the initial post-bootstrap span with standalone `state-phase-write.py ... spawn`: `plan` before its worker dispatch (normal run), or `verify` before MECHANICAL (verify-only). The bootstrap never chooses a phase, engine, branch, prompt, or agent.
|
|
99
|
+
|
|
100
|
+
### Explicit role dispatch
|
|
101
|
+
|
|
102
|
+
Use the frozen role entry and `role-config.py --state .devlyn/pipeline.state.json --role <worker|primary_judge|pair_judge> [--resolved-model <exact inherited model>]` before each affected dispatch. Keep model/effort values as individual argv elements. Replace that role's model/effort options, never append duplicates; omitted fields retain its current phase options. PLAN, BUILD_GATE, probes and SURFACE_CLOSE do not inherit worker/VERIFY profiles. A native option warning that ignores/rejects a requested field is BLOCKED even at exit0. Print requested values, source and channel before dispatch; report observed values only from validated evidence.
|
|
103
|
+
|
|
104
|
+
For a Codex worker with explicit effort, retain the exact wrapper arguments (excluding `bash` and the wrapper path) as `.devlyn/<phase>.argv.<round>.json` before launch. The wrapper's existing invocation receipt binds that array's canonical JSON hash; completion checks the requested effort against it. This adds no mutation receipt relaxation or new launcher.
|
|
105
|
+
|
|
106
|
+
For a judge with explicit model/effort, let `JUDGE_STEM=.devlyn/<engine>-judge.r<VERIFY-round>`. Retain the exact inlined prompt as `$JUDGE_STEM.prompt` and complete dispatched argv array as `$JUDGE_STEM.argv.json`. Codex uses the isolated read-only monitored route with timeout600 and redirects stdout/stderr to `$JUDGE_STEM.stdout`/`$JUDGE_STEM.stderr`. Claude uses the existing 600s `run-bounded.py` CLI route, `--tools Read,Grep,Glob --allowedTools Read,Grep,Glob --permission-mode dontAsk --setting-sources project --strict-mcp-config --mcp-config '{"mcpServers":{}}' --output-format json`; redirect raw JSON to `$JUDGE_STEM.output.json` and stderr to `$JUDGE_STEM.stderr`. Never substitute a Claude CLI worker for native Agent.
|
|
107
|
+
|
|
108
|
+
After the actual process exits, run `python3 "$DEVLYN_SHARED_DIR/judge-role-evidence.py" --devlyn-dir .devlyn --role <primary_judge|pair_judge> --exit-code <actual-exit>`. On failure preserve raw artifacts and BLOCK the source; never create successful evidence manually. On success it retains round-scoped evidence and produces canonical `.devlyn/<engine>-judge.stdout` (exact Claude result extraction, exact Codex stdout copy) for the unchanged findings collector. Merge revalidates/binds evidence and archives all bound originals, including prior rounds. Missing explicit judge evidence cannot become PASS. Unconfigured judge routes retain their existing capture format.
|
|
99
109
|
|
|
100
110
|
## PHASE 1: PLAN
|
|
101
111
|
|
|
102
112
|
Skip in verify-only mode. The heaviest phase by design — spec/criteria define non-negotiable invariants; plan formalizes how the implementation hits them.
|
|
103
113
|
|
|
104
|
-
Engine: PLAN is orchestrator-fixed and never inherits `--engine`, an executor pin, or `state.engine`; Claude Code uses a native Claude `Agent` worker, while Codex CLI / oh-my-pi use their own fresh worker per `_shared/engine-preflight.md` (PLAN-pair is **research-only** — iter-0033d/f/g; unblock conditions in PHASE 1 / iter-0033g §H; iter-0020 falsified Codex-BUILD/IMPLEMENT, NOT PLAN-pair). Before rendering, assign the current orchestrator's literal adapter name (`PLAN_ENGINE=claude`, `PLAN_ENGINE=codex`, or `PLAN_ENGINE=omp`) and assign `PLAN_MODEL=<exact model id passed to that fresh worker>`; replace the metavalue and fail closed if either assignment is empty. Write the data-only task context to `.devlyn/plan.task-context` with `PLAN_WORKING_DIR="$(pwd -P)"`: its first two lines are exactly `Working directory: $PLAN_WORKING_DIR` and `Plan output: $PLAN_WORKING_DIR/.devlyn/plan.md`, followed by the task data.
|
|
114
|
+
Engine: PLAN is orchestrator-fixed and never inherits `--engine`, an executor pin, or `state.engine`; Claude Code uses a native Claude `Agent` worker, while Codex CLI / oh-my-pi use their own fresh worker per `_shared/engine-preflight.md` (PLAN-pair is **research-only** — iter-0033d/f/g; unblock conditions in PHASE 1 / iter-0033g §H; iter-0020 falsified Codex-BUILD/IMPLEMENT, NOT PLAN-pair). Before rendering, assign the current orchestrator's literal adapter name (`PLAN_ENGINE=claude`, `PLAN_ENGINE=codex`, or `PLAN_ENGINE=omp`) and assign `PLAN_MODEL=<exact model id passed to that fresh worker>`; replace the metavalue and fail closed if either assignment is empty. Write the data-only task context to `.devlyn/plan.task-context` with `PLAN_WORKING_DIR="$(pwd -P)"`: its first two lines are exactly `Working directory: $PLAN_WORKING_DIR` and `Plan output: $PLAN_WORKING_DIR/.devlyn/plan.md`, followed by the task data. For round `PLAN_ROUND`, render adapter + `references/phases/plan.md` + task context without byte normalization to `PLAN_PROMPT_FILE=.devlyn/plan.prompt.$PLAN_ROUND`, and set `PLAN_PROMPT_SHA256="$(python3 "$DEVLYN_SHARED_DIR/phase-prompt-render.py" --adapter "$DEVLYN_SHARED_DIR/adapters/$PLAN_ENGINE.md" --canonical-body "$DEVLYN_SKILL_DIR/references/phases/plan.md" --task-context .devlyn/plan.task-context --output "$PLAN_PROMPT_FILE")"`. The renderer fails closed before writing the round-scoped prompt if either fixed header field is missing, non-absolute, or mismatches the active repository paths. Open round 0 with `state-phase-write.py ... --phase plan spawn --round 0 --engine "$PLAN_ENGINE" --model "$PLAN_MODEL" --prompt-sha256 "$PLAN_PROMPT_SHA256"`. In Claude Code, invoke the native `Agent` with its `prompt` field set to the exact `PLAN_PROMPT_FILE` bytes without appending a terminal LF (the renderer emits none), explicitly set `run_in_background: false`, and omit `mode`; this is a data field, so a path, `$(cat ...)`, or any variable-reference substitution is not the rendered prompt. If native input validation rejects this call, surface the failure without retrying a different `mode` or background shape and fail closed through the existing `BLOCKED:fresh-context-unavailable` path. The PLAN worker inherits the parent's permission context: the hands-free parent already uses native `bypassPermissions`, while a non-bypass parent's permission prompt remains the user's native boundary and is never silently escalated. A Codex CLI PLAN must pass explicit `-m "$PLAN_MODEL"` and the seven `DEVLYN_INVOCATION_*` values around `codex-monitored.sh`, using phase `plan`, `PLAN_ROUND`, `PLAN_WORKING_DIR`, `PLAN_PROMPT_FILE`, `PLAN_SESSION_FILE=.devlyn/plan.worker-session.$PLAN_ROUND.jsonl`, and `PLAN_RECEIPT_FILE=.devlyn/plan.invocation.$PLAN_ROUND.json`; invoke the wrapper as `bash "$CODEX_MONITORED_PATH" --json -C "$PLAN_WORKING_DIR" -s workspace-write -m "$PLAN_MODEL" -c sandbox_workspace_write.network_access=false <other options> "<exact PLAN_PROMPT_FILE bytes>" >"$PLAN_SESSION_FILE" 2>.devlyn/plan.stderr`, then pass that same session as `state-phase-write.py ... --engine-session-log "$PLAN_SESSION_FILE"` on completion/transition. The wrapper must write and finish the receipt around that direct capture; this state-bound receipt proves requested-model dispatch and prompt/session binding; effective model remains unknown, and a native model-reroute event blocks completion. oh-my-pi delivers the same prompt through its unchanged route. Product code records rendered intent; it never parses a transcript or intercepts, rewrites, wraps, or forces native delivery.
|
|
105
115
|
|
|
106
116
|
Subagent output (writes `.devlyn/plan.md`): file list to touch, risk list (out-of-scope expansions, ambiguous spec sections), acceptance restatement (what `## Verification` actually requires verbatim). For large work only, PLAN may add a fourth section `## Execution phases` under the conditions in `references/phases/plan.md`; a phase block missing a runnable `gate:` line disqualifies the whole section — treat the run as single-phase and log the disqualification.
|
|
107
117
|
|
|
108
|
-
State write: each PLAN receipt carries spawn-known `round`, `started_at`, `triggered_by`, `engine`, `model_requested`, and `prompt_sha256`; completion
|
|
118
|
+
State write: each PLAN receipt carries spawn-known `round`, `started_at`, `triggered_by`, `engine`, `model_requested`, and `prompt_sha256`; completion adds `completed_at`, `duration_ms`, `verdict`, nullable `model_effective`, and `output_sha256` under `references/state-schema.md`. Later mutations rehash bound output. Never edit PLAN to widen scope after completion; only the bounded PLAN re-spawn may replace its receipt.
|
|
109
119
|
|
|
110
120
|
After return:
|
|
111
121
|
1. If `.devlyn/plan.md` lists zero files → halt with verdict `BLOCKED:plan-empty`.
|
|
112
|
-
2. If risk list flags an out-of-scope expansion the user did not authorize → the sole legal corrective re-spawn is round 1: update the data-only task context, repeat render + digest
|
|
122
|
+
2. If risk list flags an out-of-scope expansion the user did not authorize → the sole legal corrective re-spawn is round 1: update the data-only task context, repeat render + digest into `.devlyn/plan.prompt.1`, open PLAN with `--round 1 --triggered-by plan` and the actual engine/model/digest, then deliver those exact prompt bytes using the same native or receipt-bound CLI call shape as round 0. A second failed plan halts; no further PLAN dispatch is authorized.
|
|
113
123
|
3. After any re-spawn above, if `state.risk_profile.risk_probes_enabled == true` and `state.risk_profile.risk_probes_explicit == false`, parse the `authorized_surface` array from the JSON block under `<!-- devlyn:authorized-surface -->`. For a well-formed string array, compute `probe_scale_small := len(authorized_surface) <= 2 AND no entry ends in "/**"`. If true, set `risk_probes_enabled = false`, leave `high_risk` unchanged, and append `auto-risk-probes demoted: plan surface small (<n> paths)` to `reasons` using the actual length. A missing or malformed block leaves probe state unchanged; BUILD_GATE owns its malformed-block failure.
|
|
114
124
|
|
|
115
125
|
## PHASE 1.5: RISK_PROBES
|
|
@@ -123,7 +133,7 @@ explicitly requested (`--risk-probes`) — an auto high-risk escalation toward a
|
|
|
123
133
|
unavailable OTHER engine was already gated off in PHASE 0 by leaving
|
|
124
134
|
`risk_probes_enabled: false`, so a single-engine high-risk run proceeds solo here.
|
|
125
135
|
|
|
126
|
-
Engine: OTHER engine from
|
|
136
|
+
Engine: OTHER engine from the legacy executor and legacy pair priority/complement; worker/VERIFY profiles do not reroute probes. Prompt body:
|
|
127
137
|
`references/phases/probe-derive.md`.
|
|
128
138
|
|
|
129
139
|
Inputs: source spec/criteria, `.devlyn/plan.md`, and repo read/search. Forbidden:
|
|
@@ -181,7 +191,8 @@ Invocation contract when OTHER engine is Codex:
|
|
|
181
191
|
|
|
182
192
|
- Invoke Codex only through the monitored wrapper path in `CODEX_MONITORED_PATH`
|
|
183
193
|
resolved from `DEVLYN_SHARED_DIR`:
|
|
184
|
-
`CODEX_MONITORED_ISOLATED=1 bash "$CODEX_MONITORED_PATH" -C "$PWD" -s workspace-write -c model_reasoning_effort=high "<probe prompt>"`.
|
|
194
|
+
`CODEX_MONITORED_ISOLATED=1 bash "$CODEX_MONITORED_PATH" -C "$PWD" -s workspace-write -c sandbox_workspace_write.network_access=false -c model_reasoning_effort=high "<probe prompt>"`.
|
|
195
|
+
Append `-c sandbox_workspace_write.network_access=true` only when a probe's visible Verification command requires a localhost service (for example, a DB test harness); never as a default.
|
|
185
196
|
Isolation keeps user config, AGENTS.md, hooks, and project rules
|
|
186
197
|
from adding hidden context, tool calls, or transcript side effects.
|
|
187
198
|
- Do not run `codex`, `codex exec`, `/Users/.../codex`, or a plugin-provided
|
|
@@ -215,14 +226,38 @@ After return:
|
|
|
215
226
|
|
|
216
227
|
Skip in verify-only mode. Constrained design judgment within PLAN's invariants. Writes code, tests, and inline doc-comments. No standalone DOCS phase — what the spec licenses is updated here, what it does not is out of scope.
|
|
217
228
|
|
|
218
|
-
Engine:
|
|
229
|
+
Engine/model/effort: frozen `role_resolution.roles.worker`; obtain validated argv additions using `role-config.py --state .devlyn/pipeline.state.json --role worker --resolved-model <existing-exact-phase-model>`. Prompt body: `references/phases/implement.md`.
|
|
230
|
+
|
|
231
|
+
For every Codex-routed IMPLEMENT, BUILD_GATE, or CLEANUP spawn, render the exact
|
|
232
|
+
prompt to `.devlyn/<phase>.prompt.<round>`, pass its SHA-256 to `state-phase-write.py
|
|
233
|
+
spawn --prompt-sha256`, and invoke only through `codex-monitored.sh` with these
|
|
234
|
+
seven variables set to the active state identity:
|
|
235
|
+
`DEVLYN_INVOCATION_{RUN_ID,PHASE,ROUND,WORKDIR,PROMPT_FILE,SESSION_FILE,RECEIPT}`.
|
|
236
|
+
The session and receipt paths are `.devlyn/<phase>.worker-session.<round>.jsonl`
|
|
237
|
+
and `.devlyn/<phase>.invocation.<round>.json`. These three paths are round-scoped
|
|
238
|
+
so a retry cannot overwrite earlier prompt/session evidence. Every invocation
|
|
239
|
+
must include `--json` and `-c sandbox_workspace_write.network_access=<true|false>`: exactly
|
|
240
|
+
`true` for BUILD_GATE and exactly `false` for PLAN, IMPLEMENT, and CLEANUP, so
|
|
241
|
+
user configuration cannot silently change the phase capability. Redirect wrapper stdout directly
|
|
242
|
+
to that session path. The wrapper rejects bypass/yolo flags and any sandbox other
|
|
243
|
+
than `workspace-write`. The receipt seals the requested model, sandbox, phase-scoped
|
|
244
|
+
network capability, prompt, terminal exit, and session digest; completion passes the
|
|
245
|
+
same canonical session via `--engine-session-log`. A missing/mismatched receipt
|
|
246
|
+
or same-round retry blocks. Respawn with a new round instead of replacing it.
|
|
219
247
|
|
|
220
248
|
State write: `phases.implement.{started_at, verdict, completed_at, duration_ms}`.
|
|
221
249
|
|
|
250
|
+
Before accepting an IMPLEMENT `PASS` / `PASS_WITH_ISSUES` completion or
|
|
251
|
+
transition, `state-phase-write.py` validates every sibling
|
|
252
|
+
`spec.expected.json.process_evidence[]` obligation for `phase: "implement"`,
|
|
253
|
+
rehashes its raw streams, and appends the validated carrier to
|
|
254
|
+
`state.process_evidence`. A missing, altered, escaping, duplicate, or
|
|
255
|
+
expectation-mismatched carrier blocks the checkpoint.
|
|
256
|
+
|
|
222
257
|
**Single-phase path** (plan.md has no `## Execution phases` section — every trivial/medium run, and any large run PLAN judged atomic): spawn IMPLEMENT once. No phase metadata of any kind appears in the prompt. After return:
|
|
223
258
|
1. `git diff --stat` — empty diff → halt with `BLOCKED:implement-empty`.
|
|
224
|
-
2.
|
|
225
|
-
3.
|
|
259
|
+
2. Checkpoint (**scoped staging** — this exact shape everywhere a pipeline commit is made): `bash -o pipefail -c 'python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --print-authorized-surface | git add --pathspec-from-file=- --pathspec-file-nul' && git commit -m "chore(pipeline): implement"`.
|
|
260
|
+
3. After the checkpoint succeeds, set `state.implement_passed_sha = git rev-parse HEAD` before downstream phases (activates `<post_implement_invariant>`).
|
|
226
261
|
|
|
227
262
|
**Phase-gated path** (plan.md has `## Execution phases` with >1 phase): definitions are the contract in plan.md; progress is routing truth in `state.phases.implement.exec = { total, current, statuses, commits }` — never route on plan.md checkbox parsing. For each phase k = 1..N:
|
|
228
263
|
1. Spawn IMPLEMENT with the standard prompt plus: this phase's plan.md block only, the current worktree as the working base (overrides the body's `base_ref.sha` framing after phase 1), a `git diff <base_ref.sha>...HEAD --stat` summary, and the prior phase's gate output.
|
|
@@ -236,7 +271,7 @@ After the final phase's gate PASS: `git diff <base_ref.sha>...HEAD --stat` — e
|
|
|
236
271
|
|
|
237
272
|
Run once iff `state.source.type == "generated"` and complexity is trivial/medium. Engine is Claude always, pair-judge-routed; executor flag/pin ignored. Per `_shared/engine-preflight.md`, unavailability skips via `surface-skip`, reports `auto_surface_close_claude_unavailable`, never BLOCKs/reroutes.
|
|
238
273
|
|
|
239
|
-
Freeze `.devlyn/surface-close.input.patch`; assemble Claude adapter + `references/phases/surface-close.md` canonical body VERBATIM + data-only Goal/patch/hashes/surface/commands; hash the bytes and spawn with `--tools "Read,Grep,Glob,Edit,Write" --dangerously-skip-permissions --
|
|
274
|
+
Freeze `.devlyn/surface-close.input.patch`; assemble Claude adapter + `references/phases/surface-close.md` canonical body VERBATIM + data-only Goal/patch/hashes/surface/commands; hash the bytes and spawn with `--tools "Read,Grep,Glob,Edit,Write" --dangerously-skip-permissions --output-format json --strict-mcp-config --mcp-config '{"mcpServers":{}}'`, omitting `--model` at SPW spawn to inherit native Claude model settings. Record the effective model from the native JSON result. Bound at 600s (native or `run-bounded.py 600 -- claude -p`), else block pre-spawn. Save raw stdout as `.devlyn/surface-close.output.json`, extract its `result` string to `.devlyn/surface-close.stdout`, retain `.devlyn/surface-close.worker-session.<round>.jsonl`, and complete with `.devlyn/surface-close.output.json` as `--engine-session-log`; non-JSON/missing result follows the existing failure path. Workers execute nothing; `surface-check` gates rows, citations, scope, and execution. Empty PASS completes; authorized delta is scoped-staged and committed. Only `BLOCKED:surface-close-adjudication-malformed` runs `surface-adjudication-recover --authorized-surface-json <array>`, whose successful rollback and transcript Edit/Write audit complete the nonterminal skip carrier and continue to BUILD_GATE. Timeout, input/prompt mismatch, attestation failure, execution-audit violation, and out-of-surface delta run `surface-rollback`, complete bare `BLOCKED`, and halt; rollback failure also halts. One shot; no `max_rounds`.
|
|
240
275
|
|
|
241
276
|
**Common post-fix checkpoint (BUILD_GATE and VERIFY):** After fix IMPLEMENT returns, increment `state.rounds.global`; scoped-stage the authorized surface and commit `chore(pipeline): implement fix round <n>`; run `python3 "$DEVLYN_SHARED_DIR/state-phase-write.py" --devlyn-dir .devlyn --phase implement durability-enforce --round <n> --origin-phase <build_gate|verify>`. Its receipt must PASS before re-entry, which enforces it again before VERIFY artifact clearing or phase spawn.
|
|
242
277
|
|
|
@@ -244,10 +279,23 @@ Freeze `.devlyn/surface-close.input.patch`; assemble Claude adapter + `reference
|
|
|
244
279
|
|
|
245
280
|
Skip in verify-only mode OR when `build-gate` in `state.bypasses`. Deterministic — same commands CI / Docker / production run.
|
|
246
281
|
|
|
247
|
-
Spawn
|
|
282
|
+
Spawn the resolved BUILD_GATE engine through the fresh-worker route in
|
|
283
|
+
`<engine_routing>` with prompt body `references/phases/build-gate.md`. A Codex
|
|
284
|
+
route must inherit CI-equivalent capabilities from the parent invocation; it
|
|
285
|
+
must not add isolation, narrow the sandbox, or widen permissions inside the
|
|
286
|
+
worker. Invoke a Codex BUILD_GATE with
|
|
287
|
+
`-s workspace-write -c sandbox_workspace_write.network_access=true`; the
|
|
288
|
+
receipt rejects a missing, false, duplicate, or non-BUILD_GATE network override
|
|
289
|
+
before the worker starts. This retains the write sandbox while allowing the
|
|
290
|
+
loopback and network operations exercised by CI-equivalent gates. If an
|
|
291
|
+
authoritative route/tool response denies a required filesystem,
|
|
292
|
+
subprocess, loopback, PTY, or network operation, preserve that denial through
|
|
293
|
+
the BUILD_GATE process-evidence manifest and complete the phase as
|
|
294
|
+
`BLOCKED:build-env-underprovisioned`, not as a product finding or substitute
|
|
295
|
+
command. The worker:
|
|
248
296
|
1. Detects language/framework via project files (`package.json`, `pyproject.toml`, etc.).
|
|
249
297
|
2. Runs language-specific gates (tsc / lint / test).
|
|
250
|
-
3. Always runs `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --include-risk-probes` (verification_commands literal-match plus `.devlyn/risk-probes.jsonl` when present). If `state.risk_profile.risk_probes_enabled == true`, the script requires `.devlyn/risk-probes.jsonl`; a missing file is a CRITICAL mechanical blocker, not a silent solo run.
|
|
298
|
+
3. Always runs `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --include-risk-probes` (verification_commands literal-match plus `.devlyn/risk-probes.jsonl` when present). If `state.risk_profile.risk_probes_enabled == true`, the script requires `.devlyn/risk-probes.jsonl`; a missing file is a CRITICAL mechanical blocker, not a silent solo run. The script routes each command through `process-evidence.py`; preserve `.devlyn/spec-verify.results.json` and its validated BUILD_GATE carrier for the state-bound archive flow.
|
|
251
299
|
4. If diff touches web-surface files: run the browser tier with the repo's available toolchain (for example Playwright or curl).
|
|
252
300
|
5. Emits `.devlyn/build_gate.findings.jsonl` + `.devlyn/build_gate.log.md`.
|
|
253
301
|
|
|
@@ -256,12 +304,13 @@ State write: `phases.build_gate.{started_at, verdict, completed_at, duration_ms,
|
|
|
256
304
|
Branch:
|
|
257
305
|
- `PASS` → PHASE 4.
|
|
258
306
|
- `FAIL` → fix loop. Spawn IMPLEMENT-engine agent with the build_gate findings as input, then run the common post-fix checkpoint with origin `build_gate`. On second FAIL with `state.rounds.global >= state.rounds.max_rounds` → halt with verdict `BLOCKED:build-gate-exhausted`.
|
|
307
|
+
- `BLOCKED:build-env-underprovisioned` → halt immediately with the denied operation, original command, and process-evidence manifest/id; do not enter the product fix loop.
|
|
259
308
|
|
|
260
309
|
## PHASE 4: CLEANUP
|
|
261
310
|
|
|
262
311
|
Skip if `cleanup` in `state.bypasses`. Task-scoped pass.
|
|
263
312
|
|
|
264
|
-
Engine:
|
|
313
|
+
Engine/model/effort: the same frozen worker profile, with validated role-config argv additions. Prompt body: `references/phases/cleanup.md`. Allowlist enforced post-spawn:
|
|
265
314
|
- Tooling artifacts the spec did not list as deliverables (`test-results/`, `playwright-report/`, `.last-run.json`, coverage HTML).
|
|
266
315
|
- Dead code added by this diff (not pre-existing dead code).
|
|
267
316
|
- Doc references whose target this diff renamed or removed.
|
|
@@ -273,53 +322,62 @@ State write: `phases.cleanup.{started_at, verdict, completed_at, duration_ms, pr
|
|
|
273
322
|
After return:
|
|
274
323
|
1. Run `git diff --name-only <pre_sha>` and compare new untracked files against `.devlyn/untracked.baseline` (`.devlyn/` exempt) — any path outside the cleanup allowlist (including a cleanup-created untracked file) → revert to `pre_sha`, delete the unauthorized untracked file, and emit `invariant.cleanup-out-of-scope` into `.devlyn/cleanup.findings.jsonl`.
|
|
275
324
|
2. If allowlist honored and diff non-empty: `git add -u && git commit -m "chore(pipeline): cleanup"`.
|
|
276
|
-
3.
|
|
325
|
+
3. Transition CLEANUP→VERIFY through `<transition_protocol>` with `--post-sha "$(git rev-parse HEAD)"` (also when the diff was empty), preserving completion/session arguments and the next-phase identity below.
|
|
277
326
|
|
|
278
327
|
## PHASE 5: VERIFY (fresh subagent, findings-only)
|
|
279
328
|
|
|
280
|
-
Independent quality layer.
|
|
329
|
+
Independent quality layer. MECHANICAL runs first on the post-CLEANUP code. Each
|
|
330
|
+
JUDGE is then **spawned with empty conversation context** — no carry-over from
|
|
331
|
+
PHASE 1-4 — and receives only `spec.md` (or
|
|
332
|
+
`.devlyn/criteria.generated.md`), sibling `spec.expected.json`, the cumulative
|
|
333
|
+
diff, the source hash, `.devlyn/spec-verify.results.json`, and the validated
|
|
334
|
+
VERIFY process-evidence manifest/raw streams named by that results carrier. The
|
|
335
|
+
fresh-context spawn plus immutable evidence inputs are the structural guarantee
|
|
336
|
+
of independence.
|
|
337
|
+
|
|
338
|
+
Before MECHANICAL, open VERIFY with frozen `role_resolution.roles.primary_judge`; without a profile it follows the legacy executor. Use that engine/model and the current round/trigger metadata in the predecessor's `<transition_protocol>` handoff, including bypass and repair paths; initial verify-only uses PHASE 0's standalone spawn. This records the phase span, not a judge invocation, and requires no rendered judge prompt or prompt digest.
|
|
281
339
|
|
|
282
340
|
Two sub-phases:
|
|
283
341
|
|
|
284
|
-
1. **MECHANICAL** (deterministic): re-run `SPEC_VERIFY_PHASE=verify_mechanical SPEC_VERIFY_FINDINGS_FILE=verify-mechanical.findings.jsonl SPEC_VERIFY_FINDING_PREFIX=VERIFY-MECH python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --include-risk-probes` against the post-CLEANUP code (independent of BUILD_GATE's earlier run). If `state.risk_profile.risk_probes_enabled == true`, missing `.devlyn/risk-probes.jsonl` is a CRITICAL mechanical blocker. This emits `.devlyn/verify-mechanical.findings.jsonl`
|
|
342
|
+
1. **MECHANICAL** (deterministic, orchestrator-owned): re-run `SPEC_VERIFY_PHASE=verify_mechanical SPEC_VERIFY_FINDINGS_FILE=verify-mechanical.findings.jsonl SPEC_VERIFY_FINDING_PREFIX=VERIFY-MECH python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --include-risk-probes` against the post-CLEANUP code (independent of BUILD_GATE's earlier run). If `state.risk_profile.risk_probes_enabled == true`, missing `.devlyn/risk-probes.jsonl` is a CRITICAL mechanical blocker. This emits `.devlyn/verify-mechanical.findings.jsonl` plus `.devlyn/spec-verify.results.json` with the VERIFY process-evidence carrier. Validate that carrier and all named raw streams before JUDGE spawn; a verdict-binding blocker skips both judges.
|
|
285
343
|
|
|
286
|
-
2. **JUDGE** (fresh-context Agent): grade the diff against the spec on rubric axes (spec compliance, scope, quality, consistency). Split each Requirement into binding clauses and trace code-order counterexamples; a passing
|
|
344
|
+
2. **JUDGE** (fresh-context Agent): grade the diff against the spec on rubric axes (spec compliance, scope, quality, consistency). Split each Requirement into binding clauses and trace code-order counterexamples; a passing sealed result proves only the case it exercises, not neighboring `once` / `regardless` / `duplicate` / auth-order / rollback invariants. Respect scope qualifiers such as `inside a warehouse`, `per resource`, `for this line`, and `after validation`; do not widen a scoped clause into a global invariant, and compose multiple ordering rules in the stated order. For stateful flows, explicitly trace failed-operation rollback and the next entity's state. For high-complexity specs, review at least one interaction counterexample that combines ordering/priority with failure handling and state mutation against the implementation and sealed MECHANICAL results. JUDGE never executes literal verification, lint, test, build, risk-probe, or newly invented interaction commands; missing executable coverage is a finding against the expected contract/risk probes. After MECHANICAL passes, `--no-pair` continues solo with `user_no_pair`. Otherwise use the frozen pair profile/priority resolved against the primary. An unavailable OTHER engine or failed authentication blocks `--pair-verify` and configured pair role/priority pins; only an unconfigured automatic route records `auto_pair_other_engine_unavailable` and continues solo.
|
|
287
345
|
|
|
288
|
-
When the OTHER engine is available, persist `pair_trigger` before spawn with `pair.default` plus every applicable outcome-independent reason, then dispatch both judges concurrently against the same frozen diff via foreground parallel dispatch, never background shells. A primary blocker never cancels the pair judge. If foreground parallel dispatch is unavailable, run the same two required judges sequentially; this changes dispatch shape only and never permits outcome-dependent escalation or a pair skip.
|
|
346
|
+
When the OTHER engine is available, persist `pair_trigger` before spawn with `pair.default` plus every applicable outcome-independent reason, then dispatch both judges concurrently against the same frozen diff and sealed MECHANICAL evidence via foreground parallel dispatch, never background shells. A primary blocker never cancels the pair judge. If foreground parallel dispatch is unavailable, run the same two required judges sequentially; this changes dispatch shape only and never permits outcome-dependent escalation or a pair skip.
|
|
289
347
|
|
|
290
348
|
Keep the existing reasons as telemetry: `mode.verify-only`, `mode.pair-verify`, `complexity.high`, `complexity.large`, `spec.complexity.high`, `spec.complexity.large`, `spec.solo_headroom_hypothesis`, `risk.high`, `risk_probes.enabled`, `risk_probes.present`, `coverage.failed`, `mechanical.warning`, and `judge.warning`. Append the outcome-dependent reasons only after both judges return. Eligible schema-v3 state must include `pair.default`; all other applicable reasons remain completeness-checked. Malformed risk or trigger state BLOCKs VERIFY.
|
|
291
349
|
|
|
292
|
-
Pair-mode JUDGE: spawn a second Agent with the OTHER engine's adapter; the second judge is a bounded adversarial complement, not a duplicate broad audit. The primary judge owns broad coverage; pair-JUDGE
|
|
350
|
+
Pair-mode JUDGE: spawn a second Agent with the OTHER engine's adapter; the second judge is a bounded adversarial complement, not a duplicate broad audit. The primary judge owns broad coverage; pair-JUDGE reviews the two highest-risk explicit `## Verification` bullets that cross state mutation, all-or-nothing rollback, ordering, idempotency, auth, or error-priority clauses. If the spec includes a solo-headroom hypothesis, one targeted review must use the hypothesis's backticked observable command as the exact anchor and inspect its complete sealed result (exit/stdout/stderr plus the full parsed output object). It must not read `.claude/skills`, `.codex/skills`, `CLAUDE.md`, `AGENTS.md`, or other harness docs unless the orchestrator pasted a specific excerpt into the prompt. It may use only the spec, diff, implementation files, tests, and sealed MECHANICAL evidence. It performs at most two targeted reviews before first output; inspection remains read-only. When the spec names exact keys, row shapes, JSON object shape, or an exact error body, pair-JUDGE compares the sealed parsed key sets/deep equality so aliased keys, missing keys, and extra keys are verdict-binding failures. For priority/stateful specs, trace code order for an earlier input entity that would succeed under input-order processing, a later higher-priority entity that consumes or blocks the critical resource, and a failure/blocked/rollback edge that determines a later entity's state. Scope qualifiers are binding: pair-JUDGE must not reinterpret `inside a warehouse`, `per resource`, or line-scoped rules as global rules. When both priority ordering and rollback/blocked-interval behavior appear in the spec, this dominance-loss review comes first: trace whether the earlier lower-priority entity loses because the later higher-priority entity is processed first, whether a failed/blocked middle entity leaves later state intact, and whether the sealed evidence covers complete accepted/scheduled and rejected output ordering. Missing executable coverage is a verdict-binding coverage finding; JUDGE does not invent or run the missing scenario. Pair-JUDGE output: emit JSONL findings then a bare terminal verdict line, or emit only `PASS` when clean. `_shared/judge-output-parser.py` is the single acceptance rule for pair output: JSONL findings, then a `# SUMMARY {json}` line or a bare verdict line (`PASS` alone when clean); it ignores bare code-fence lines and unwraps the registered Codex JSON envelope including its narrated-preamble recovery (iter-0082), and binds a whole-message NDJSON capture only through its uniquely attested terminal `end_turn` assistant message (iter-0106); every other non-empty line blocks the pair source. Both judgments merge with the rule "any HIGH/CRITICAL finding is verdict-binding; any MEDIUM with literal `verdict_binding: true` is also binding, regardless of confidence." Demonstrated applicable mandatory-clause violations, including unmet new requirements and incorrect customer documentation, must use one of those binding forms. Apply the canonical VERIFY clause/evidence and advisory boundaries; impact, rarity or pre-existing origin alone does not excuse an applicable violation. Cross-model disagreement on advisory lower-severity findings is logged but does not change the verdict. A primary JUDGE verdict-binding finding on a concurrent run does not cancel or discard the pair-JUDGE; both finding sets join the same fix round, and the merged verdict remains the worst source verdict without vote counting. `primary_judge_blocker` is parser-recognized only for archived v2.0 replay; new runs never write it. A verdict-binding MECHANICAL blocker still skips both judges and records `pair_judge: null`.
|
|
293
351
|
|
|
294
|
-
If the OTHER engine is unavailable, automatic VERIFY records
|
|
295
|
-
`auto_pair_other_engine_unavailable` and continues solo. Explicit `--pair-verify
|
|
296
|
-
instead
|
|
352
|
+
If the OTHER engine is unavailable, only unconfigured automatic VERIFY records
|
|
353
|
+
`auto_pair_other_engine_unavailable` and continues solo. Explicit `--pair-verify`,
|
|
354
|
+
pair profiles and priority pins instead set VERIFY to `BLOCKED:<engine>-unavailable`, preserves the failed
|
|
297
355
|
check, and prints setup guidance: install/configure the missing CLI, complete
|
|
298
356
|
its auth/login flow, verify `<engine> --version`, then rerun. `--no-pair` is only
|
|
299
357
|
for an intentional solo run.
|
|
300
358
|
|
|
301
|
-
Findings written to `.devlyn/verify.findings.jsonl`. **VERIFY agents
|
|
359
|
+
Findings are written to `.devlyn/verify.findings.jsonl`. **VERIFY agents may inspect authorized source, diff and sealed evidence using native read/search tools or non-mutating shell commands. They must not mutate files or execute verification, lint, test, build, probe or invented-scenario commands.** Capture the primary reply as `.devlyn/<primary-engine>-judge.stdout` and its stderr sibling; never use the generic `.devlyn/verify-judge.stdout`. A Codex primary-JUDGE uses the distinct route `CODEX_MONITORED_ISOLATED=1 CODEX_MONITORED_TIMEOUT_SEC=600 bash "$CODEX_MONITORED_PATH" -C "$PWD" -s read-only -c model_reasoning_effort=high "<primary prompt>" >.devlyn/codex-judge.stdout 2>.devlyn/codex-judge.stderr`: omit `-m` only without an explicit model; an explicit profile supplies the validated model/effort additions below. Omit bypass flags and capture both streams directly without pipes. On primary exit 124, write `.devlyn/verify.primary.timeout.json` with exactly `{"engine": "<resolved-primary-engine>", "budget_seconds": 600}` before merge. A malformed marker BLOCKs; a valid marker preserves canonical primary findings and floors `judge` at `BLOCKED` even when output is empty, never `PASS`, a solo verdict, or pair-style `TIMEOUT`. Without the marker, existing required-primary failure behavior is unchanged. For Codex pair-JUDGE, keep the read-only `codex-monitored.sh` route with `CODEX_MONITORED_ISOLATED=1 CODEX_MONITORED_TIMEOUT_SEC=600` and `-c model_reasoning_effort=medium`, no `tail`/`head`/`grep` pipes, and direct stdout/stderr capture; every other resolved OTHER engine follows `_shared/adapters/<name>.md` `## Invocation`. Capture the pair reply as `.devlyn/<other-engine>-judge.stdout`, then run `python3 "$DEVLYN_SHARED_DIR/collect-codex-findings.py" --devlyn-dir "<abs repo>/.devlyn" --stdout-file <other-engine>-judge.stdout` before merge. Raw stdout is diagnostic-only. A non-zero collector exit must write no canonical `.devlyn/verify.pair.findings.jsonl`; set the pair source to `BLOCKED` for `verify.pair.emission-contract` and never merge as though unparsed stdout were only a diagnostic. VERIFY prompts remain bounded, read-only, and must not ask the judge to edit `.devlyn`. On pair-JUDGE subprocess exit 124, write `.devlyn/verify.pair.timeout.json` with `{"engine": "<resolved name>", "budget_seconds": 600}` before merge. With that marker, a capture the parser rejects solely because its message stream was truncated before the terminal result stays `TIMEOUT`; every other parser rejection stays `BLOCKED`. The orchestrator records `phases.verify.judge_durations_ms: {judge: <int>, pair_judge: <int|null>}` as it collects each judge result; durations never nest inside `sub_verdicts`. After primary and pair findings are written, run `python3 "$DEVLYN_SHARED_DIR/verify-merge-findings.py" --write-state`; it rehashes and binds the VERIFY process-evidence carrier before merging. Branch only on the merged `state.phases.verify.verdict`; a HIGH/CRITICAL finding from either judge must mechanically become `NEEDS_WORK`. Never write `.devlyn/verify-merged.findings.jsonl` or `.devlyn/verify-merge.summary.json` by hand; `verify-merge-findings.py` is their only writer. State write: `phases.verify.{started_at, verdict, completed_at, duration_ms, judge_durations_ms: {judge, pair_judge}, sub_verdicts: {mechanical, judge, pair_judge?}, artifacts}`.
|
|
302
360
|
|
|
303
361
|
Branch:
|
|
304
362
|
- `PASS` → PHASE 6.
|
|
305
|
-
- `PASS_WITH_ISSUES` (
|
|
363
|
+
- `PASS_WITH_ISSUES` (no verdict-binding findings) → PHASE 6 with banner.
|
|
306
364
|
- `NEEDS_WORK` / `BLOCKED` → fix loop with `triggered_by: "verify"`. Spawn IMPLEMENT-engine agent with the verify findings, then run the common post-fix checkpoint with origin `verify`. Second `NEEDS_WORK` → halt with verdict `BLOCKED:verify-exhausted`.
|
|
307
365
|
|
|
308
366
|
## PHASE 6: FINAL REPORT + ARCHIVE
|
|
309
367
|
|
|
310
|
-
|
|
368
|
+
Open the `final_report` span through the predecessor's `state-phase-write.py --devlyn-dir .devlyn --phase <predecessor> transition --next-phase final_report --next-round 0 …` on a direct handoff; after a halt, use standalone `state-phase-write.py --devlyn-dir .devlyn --phase final_report spawn --round 0` only if the span is unopened.
|
|
311
369
|
|
|
312
370
|
1. Kill any dev server PHASE 3 left running.
|
|
313
371
|
|
|
314
|
-
2. **FINISH GATE** — run `python3 "$DEVLYN_SHARED_DIR/finish-gate.py"`; branch only on exit code
|
|
372
|
+
2. **FINISH GATE** — run `python3 "$DEVLYN_SHARED_DIR/finish-gate.py"`; branch only on exit code: 0 → clean; 1 or 2 → `BLOCKED:finish-gate-unclean`, and report the `.devlyn/finish-gate.findings.jsonl` listing, including reverted paths. Offenders exit 2 even when every revert succeeded — a silent revert must never ride an exit-0 pass.
|
|
315
373
|
|
|
316
374
|
3. **Terminal verdict** — derive from `state.phases.{plan, implement, surface_close, build_gate, cleanup, verify}.verdict` per the precedence rules in `references/state-schema.md#terminal-verdict`. Verify-only mode short-circuits to `state.phases.verify.verdict`.
|
|
317
375
|
|
|
318
|
-
4. **Render report** —
|
|
376
|
+
4. **Render report to `.devlyn/final-report.md` before completion** — first line exactly `<!-- devlyn:final-report run_id=<current state.run_id> -->`, once only, followed by a nonempty report body. Sections: header (run_id, engine, mode, verdict, wall-time), per-phase summary (including SURFACE_CLOSE run or skip), pair/risk-probe status, findings table (verify + finish-gate findings), follow-up notes (the explicit line `pipeline continued to BUILD_GATE — surface_close_rolled_back_adjudication_malformed` when `continued_after_block` is set, any large-mode `## Assumptions` block, any pair-judge TIMEOUT (headline: solo verdict after pair TIMEOUT), any `--no-pair` / `--no-risk-probes` opt-out, any engine setup guidance after BLOCKED, `/devlyn:ideate` guidance after `BLOCKED:solo-headroom-hypothesis-required` that asks for the visible behavior `solo_claude` is expected to miss, and `/devlyn:ideate` guidance after `BLOCKED:solo-ceiling-avoidance-required` that asks for the concrete difference from rejected or solo-saturated controls such as `S2`-`S6`). User-facing text may follow archive; it cannot substitute for this file.
|
|
319
377
|
|
|
320
|
-
5.
|
|
378
|
+
5. Complete the span with `state-phase-write.py --devlyn-dir .devlyn --phase final_report complete --verdict <bare enum> --log-file .devlyn/final-report.md` — the enum class of the terminal verdict (`BLOCKED:<reason>` → `BLOCKED`; `NEEDS_WORK` / `PASS_WITH_ISSUES` / `PASS` unchanged) — BEFORE archive runs (archive prune skips runs whose `final_report.verdict` is null). The writer validates the canonical nonsymlink regular file, current run marker and nonempty body, then binds its exact bytes; validation failure leaves the phase open. Never hand-edit lifecycle fields in `pipeline.state.json` (`references/state-schema.md` § Write protocol).
|
|
321
379
|
|
|
322
|
-
6. **Archive** — invoke the deterministic script: `python3 "$DEVLYN_SHARED_DIR/archive_run.py"`. The script reads `run_id` from `.devlyn/pipeline.state.json`, moves the per-run artifact set (`PER_RUN_PATTERNS`
|
|
380
|
+
6. **Archive** — invoke the deterministic script: `python3 "$DEVLYN_SHARED_DIR/archive_run.py"`. The script reads `run_id` from `.devlyn/pipeline.state.json`, moves the static per-run artifact set (`PER_RUN_PATTERNS` remains the single ownership list) plus every state-bound process-evidence manifest/raw stream into `.devlyn/runs/<run_id>/`, preserves evidence-relative layout, and rehashes bound bytes before any move. An unsafe/missing/altered evidence path or destination collision reports archive failure without changing the already-derived product verdict. It then best-effort prunes to the last 10 completed runs. Archive must run; running this step as deterministic-script-not-prose ensures the move actually happens (iter-0033a Smoke 3 caught a case where the agent claimed archive ran without moving the files).
|
|
323
381
|
|
|
324
382
|
## State management
|
|
325
383
|
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Outer-loop commit boundary
|
|
2
|
+
|
|
3
|
+
A full `/devlyn:resolve` run starts only from committed owner inputs. Keep each
|
|
4
|
+
commit scoped: no unrelated paths or queue-item changes.
|
|
5
|
+
|
|
6
|
+
1. Before an item's first full run, commit the current queue-item delta, when
|
|
7
|
+
applicable, together with its linked spec bundle. For conversational work,
|
|
8
|
+
commit the agreed spec bundle before the full run.
|
|
9
|
+
2. When findings require an accepted verification amendment, commit only that
|
|
10
|
+
linked spec-bundle amendment before rerunning the item.
|
|
11
|
+
3. After a terminal result, commit the current item's `[x]` or `[F]` queue
|
|
12
|
+
transition before starting the next pending item.
|
|
13
|
+
|
|
14
|
+
Drain strictly serially. Bootstrap and resolve never make these owner commits;
|
|
15
|
+
the outer-loop driver owns them.
|
|
@@ -6,6 +6,35 @@ Per-engine adapter header is prepended at runtime. BUILD_GATE is mechanical / de
|
|
|
6
6
|
Run language-specific gates and the spec literal-match verification. Emit findings; the orchestrator's fix loop consumes them.
|
|
7
7
|
</role>
|
|
8
8
|
|
|
9
|
+
<capability_contract>
|
|
10
|
+
A Codex BUILD_GATE runs on the parent-provided CI-equivalent route. Required
|
|
11
|
+
filesystem, subprocess, loopback, PTY, and network capabilities are part of the
|
|
12
|
+
gate input, not permissions this worker may widen.
|
|
13
|
+
|
|
14
|
+
If the parent route or an authoritative tool response explicitly denies a
|
|
15
|
+
required operation before the command can produce product output:
|
|
16
|
+
|
|
17
|
+
1. Preserve the exact original command, denied operation, and raw denial through
|
|
18
|
+
the shared `process-evidence.py` manifest for this BUILD_GATE run/round. Use
|
|
19
|
+
its `capability_denied` classification; a matching stderr phrase is never
|
|
20
|
+
sufficient classification.
|
|
21
|
+
2. Stop with `BLOCKED:build-env-underprovisioned`, citing
|
|
22
|
+
`operation=<filesystem|subprocess|loopback|pty|network>`, the command, and the
|
|
23
|
+
manifest path plus evidence id.
|
|
24
|
+
3. Do not emit a product finding, run a substitute command, narrow the command,
|
|
25
|
+
retry it on another route, or treat the denial as test/lint output.
|
|
26
|
+
|
|
27
|
+
The state writer derives the phase verdict floor from this sealed manifest. A
|
|
28
|
+
capability denial can complete only as `BLOCKED`; a failed product expectation
|
|
29
|
+
cannot complete as `PASS` or `PASS_WITH_ISSUES` even if a mutable results or
|
|
30
|
+
findings file claims otherwise.
|
|
31
|
+
|
|
32
|
+
An executed command that exits nonzero remains a genuine product result even if
|
|
33
|
+
its stdout/stderr contains text such as `permission denied` or `operation not
|
|
34
|
+
permitted`. Preserve the existing finding rules below. Only explicit route/tool
|
|
35
|
+
capability evidence produces the blocker.
|
|
36
|
+
</capability_contract>
|
|
37
|
+
|
|
9
38
|
<detection>
|
|
10
39
|
Detect the project shape from files in `state.base_ref.sha`:
|
|
11
40
|
|
|
@@ -22,7 +51,7 @@ Run in this order; each emits findings into `.devlyn/build_gate.findings.jsonl`:
|
|
|
22
51
|
1. **Type check** (TypeScript / mypy / etc.). Each error → one finding, severity `HIGH`, rule `correctness.type-check`.
|
|
23
52
|
2. **Lint**. Run a recognized configured language linter (for example, ESLint with a config, `ruff`, or `clippy`) and/or the exact `package.json` `scripts.lint` command. A declared specialized `lint:*` script (for example, `lint:json`) is not the general language lint gate on the strength of its declaration alone. If the spec's verification commands explicitly name such a specialized command, step 4 executes it and preserves its real failure severity. If no linter is applicable, log an explicit SKIP in `.devlyn/build_gate.log.md` and emit zero lint findings. Each error → finding, severity `MEDIUM`, rule `quality.lint`. Warnings stay LOW unless the spec elevates them.
|
|
24
53
|
3. **Test suite** (npm test / pytest / go test / cargo test). Each failing test → finding, severity `HIGH`, rule `correctness.test-failure`. Include the failing test's file:line and the assertion.
|
|
25
|
-
4. **Spec literal verification + risk probes**: `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --include-risk-probes`. The script self-stages from sibling `spec.expected.json` next to `state.source.spec_path`, or the legacy inline carrier when the sibling is absent; benchmark-prestaged `.devlyn/spec-verify.json` still wins. It appends `.devlyn/risk-probes.jsonl` when present, and requires that file when `state.risk_profile.risk_probes_enabled == true`. Malformed `state.risk_profile` is also CRITICAL because it can hide enabled risk probes. When probes are enabled or `.devlyn/risk-probes.jsonl` exists, missing or mismatched top-level `state.risk_probes_digest` emits `correctness.risk-probe-integrity` CRITICAL; the digest binds `.devlyn/risk-probes.jsonl` plus referenced `.devlyn/probes/` script bytes. Command or risk-probe mismatch → CRITICAL finding. Missing required risk probes, missing/malformed generated carrier, or malformed sibling expected file → `correctness.spec-verify-malformed` CRITICAL. The same invocation also enforces PLAN's declared `authorized_surface` (the json block under `.devlyn/plan.md`'s `<!-- devlyn:authorized-surface -->` section, located by that sentinel regardless of the heading's language) against this run's diff and untracked delta: any changed path or created-during-run untracked path outside the declared surface → `scope.out-of-scope-file` CRITICAL (`.devlyn/` exempt) — the only sanctioned fix is removing the file, never widening the declared surface, since that would let the same worker that leaked scope self-authorize the leak; missing sentinel, malformed `plan.md`/surface block, or missing `.devlyn/untracked.baseline` → `scope.authorized-surface-malformed` CRITICAL. This scope check runs only here (BUILD_GATE), not when VERIFY MECHANICAL reuses this script post-CLEANUP, because CLEANUP's own allowlist legitimately touches paths PLAN never declared.
|
|
54
|
+
4. **Spec literal verification + risk probes**: `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --include-risk-probes`. The script routes every literal command and risk probe through `process-evidence.py`, preserving exact command, outcome, raw stdout/stderr, and digests in the BUILD_GATE run/round manifest. It self-stages from sibling `spec.expected.json` next to `state.source.spec_path`, or the legacy inline carrier when the sibling is absent; benchmark-prestaged `.devlyn/spec-verify.json` still wins. It appends `.devlyn/risk-probes.jsonl` when present, and requires that file when `state.risk_profile.risk_probes_enabled == true`. Malformed `state.risk_profile` is also CRITICAL because it can hide enabled risk probes. When probes are enabled or `.devlyn/risk-probes.jsonl` exists, missing or mismatched top-level `state.risk_probes_digest` emits `correctness.risk-probe-integrity` CRITICAL; the digest binds `.devlyn/risk-probes.jsonl` plus referenced `.devlyn/probes/` script bytes. Command or risk-probe mismatch → CRITICAL finding. Missing required risk probes, missing/malformed generated carrier, or malformed sibling expected file → `correctness.spec-verify-malformed` CRITICAL. The same invocation also enforces PLAN's declared `authorized_surface` (the json block under `.devlyn/plan.md`'s `<!-- devlyn:authorized-surface -->` section, located by that sentinel regardless of the heading's language) against this run's diff and untracked delta: any changed path or created-during-run untracked path outside the declared surface → `scope.out-of-scope-file` CRITICAL (`.devlyn/` exempt) — the only sanctioned fix is removing the file, never widening the declared surface, since that would let the same worker that leaked scope self-authorize the leak; missing sentinel, malformed `plan.md`/surface block, or missing `.devlyn/untracked.baseline` → `scope.authorized-surface-malformed` CRITICAL. This scope check runs only here (BUILD_GATE), not when VERIFY MECHANICAL reuses this script post-CLEANUP, because CLEANUP's own allowlist legitimately touches paths PLAN never declared.
|
|
26
55
|
5. **Browser** (only when diff touches `*.tsx`, `*.jsx`, `*.vue`, `*.svelte`, `page.*`, `layout.*`, `route.*`, `*.css`, `*.html`): start the dev server and run the repo's existing browser checks, or a minimal curl/HTML check when no browser test harness exists. Each failed check → finding, severity `HIGH`, rule `correctness.browser-flow-failed`.
|
|
27
56
|
|
|
28
57
|
Append all findings; do not stop on the first failure.
|
|
@@ -31,7 +60,7 @@ Append all findings; do not stop on the first failure.
|
|
|
31
60
|
<output>
|
|
32
61
|
- `.devlyn/build_gate.findings.jsonl` — JSONL stream, one finding per line. Schema: `{id, rule_id, severity, file, line, message, fix_hint, criterion_ref}`.
|
|
33
62
|
- `.devlyn/build_gate.log.md` — human-readable summary of which gates ran and their raw output.
|
|
34
|
-
- Report
|
|
63
|
+
- Report `PASS` if zero CRITICAL/HIGH findings, `FAIL` for genuine product findings, or `BLOCKED:build-env-underprovisioned` only for the evidence-backed capability path above. A capability blocker leaves no product finding. Do not edit `pipeline.state.json` yourself — the orchestrator records it via `state-phase-write.py` from these artifacts.
|
|
35
64
|
</output>
|
|
36
65
|
|
|
37
66
|
<quality_bar>
|
|
@@ -14,7 +14,8 @@ You execute the plan. Constrained design judgment within PLAN's invariants — w
|
|
|
14
14
|
|
|
15
15
|
<output>
|
|
16
16
|
- Code changes implementing every Requirement. Verify with `git diff`.
|
|
17
|
-
- Tests added or updated for changed behavior. Run the
|
|
17
|
+
- Tests added or updated for changed behavior. Run the focused development tests needed to establish that behavior; BUILD_GATE and VERIFY own the post-implementation full suite.
|
|
18
|
+
- For every sibling `spec.expected.json.process_evidence[]` item whose `phase` is `implement`, run `python3 "$DEVLYN_SHARED_DIR/process-evidence.py" --devlyn-dir .devlyn run --phase implement --id '<id>'`. The runner must report `expectation_met: true`; cite its manifest path in the phase reply.
|
|
18
19
|
- For each criterion satisfied, set `state.criteria[i].status: "implemented"` with an `evidence` record `{"file": "...", "line": N, "note": "brief"}`.
|
|
19
20
|
- Report your verdict in this reply: `PASS` on success; `BLOCKED` if a criterion cannot be satisfied (missing external dep, blocking ambiguity in the spec) — never silently `pending`. Do not edit `pipeline.state.json` yourself — the orchestrator records it via `state-phase-write.py`.
|
|
20
21
|
</output>
|
|
@@ -22,7 +23,7 @@ You execute the plan. Constrained design judgment within PLAN's invariants — w
|
|
|
22
23
|
<quality_bar>
|
|
23
24
|
- Spec is the contract. The plan is the path. If they disagree, surface the conflict and follow the spec.
|
|
24
25
|
- Bugs: write the failing test first, then fix. Features: follow existing patterns, then write tests. Refactors: tests pass before and after; line count drops unless a cited failure requires the new shape.
|
|
25
|
-
-
|
|
26
|
+
- Do not execute the spec's `verification_commands` in IMPLEMENT; VERIFY MECHANICAL executes those literal commands once after CLEANUP. Declared IMPLEMENT process obligations, including red-first commands, run only through the shared evidence runner above.
|
|
26
27
|
- Tooling-generated artifacts (`test-results/`, `playwright-report/`, `.last-run.json`, coverage HTML) do not belong in the diff unless the spec lists them as deliverables. Configure tools to emit to gitignored paths.
|
|
27
28
|
- Existing tests are contract. Do not replace real HTTP / filesystem / subprocess calls with mocks. Do not skip or disable tests. Do not reduce assertion count on behavior still in scope.
|
|
28
29
|
- Files not in PLAN's list are off-limits. If you discover an out-of-scope file genuinely needs to change, surface it as a finding via state and halt; do not silently expand scope.
|