devlyn-cli 1.14.0 → 1.15.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (22) hide show
  1. package/CLAUDE.md +28 -149
  2. package/config/skills/devlyn:auto-resolve/SKILL.md +165 -515
  3. package/config/skills/devlyn:auto-resolve/evals/evals.json +21 -0
  4. package/config/skills/devlyn:auto-resolve/evals/task-doctor-subcommand.md +42 -0
  5. package/config/skills/devlyn:auto-resolve/references/build-gate.md +36 -22
  6. package/config/skills/devlyn:auto-resolve/references/engine-routing.md +43 -165
  7. package/config/skills/devlyn:auto-resolve/references/findings-schema.md +103 -0
  8. package/config/skills/devlyn:auto-resolve/references/phases/phase-1-build.md +54 -0
  9. package/config/skills/devlyn:auto-resolve/references/phases/phase-2-evaluate.md +45 -0
  10. package/config/skills/devlyn:auto-resolve/references/phases/phase-3-critic.md +84 -0
  11. package/config/skills/devlyn:auto-resolve/references/pipeline-routing.md +114 -0
  12. package/config/skills/devlyn:auto-resolve/references/pipeline-state.md +201 -0
  13. package/config/skills/devlyn:auto-resolve/scripts/archive_run.py +104 -0
  14. package/config/skills/devlyn:auto-resolve/scripts/terminal_verdict.py +96 -0
  15. package/config/skills/devlyn:ideate/SKILL.md +12 -64
  16. package/config/skills/devlyn:ideate/references/codex-critic-template.md +42 -0
  17. package/config/skills/devlyn:preflight/SKILL.md +25 -40
  18. package/config/skills/devlyn:preflight/references/auditors/code-auditor.md +6 -10
  19. package/config/skills/devlyn:reap/SKILL.md +104 -0
  20. package/config/skills/devlyn:reap/scripts/reap.sh +129 -0
  21. package/config/skills/devlyn:reap/scripts/scan.sh +116 -0
  22. package/package.json +5 -1
@@ -1,602 +1,252 @@
1
1
  ---
2
2
  name: devlyn:auto-resolve
3
- description: Fully automated build-evaluate-polish pipeline for any task type — bug fixes, new features, refactors, chores, and more. Use this as the default starting point when the user wants hands-free implementation with zero human intervention. Runs the full cycle — build, evaluate, fix loop, simplify, review, clean, docs — as a single command. Use when the user says "auto resolve", "build this", "implement this feature", "fix this", "run the full pipeline", "refactor this", or wants to walk away and come back to finished work.
3
+ description: Fully automated build-evaluate-ship pipeline for any task type — bug fixes, new features, refactors, chores. Use this as the default starting point when the user wants hands-free implementation with zero human intervention. Runs a minimal goal-driven loop — build, evaluate, fix, critic, docs — as a single command. Use when the user says "auto resolve", "build this", "implement this feature", "fix this", "run the full pipeline", "refactor this", or wants to walk away and come back to finished work.
4
4
  ---
5
5
 
6
- Fully automated resolve-evaluate-polish pipeline. One command, zero human intervention. Spawns a subagent for each phase, uses file-based handoff between phases, and loops on evaluation feedback until the work passes or max rounds are reached.
6
+ Orchestrator for the hands-free implementation pipeline. One subagent per phase, file-based handoff, unified fix loop on evaluation feedback until the work passes or `max_rounds` is reached. The orchestrator itself does not write code — it parses input, spawns phases, reads handoff artifacts, runs git commands, branches on verdicts, and emits the final report.
7
7
 
8
8
  <pipeline_config>
9
9
  $ARGUMENTS
10
10
  </pipeline_config>
11
11
 
12
- <pipeline_workflow>
13
-
14
12
  <orchestrator_context>
15
- This pipeline is long-horizon agentic work. As the orchestrator, you spawn many subagents and read their handoff files; your own context grows over the run.
16
-
17
- - Your context window is auto-compacted as it approaches its limit, so do not stop tasks early due to token-budget concerns. Keep the run going.
18
- - All durable state lives in `.devlyn/*.md` (done-criteria, BUILD-GATE, EVAL-FINDINGS, BROWSER-RESULTS, CHALLENGE-FINDINGS) and in git commits. If your context is cleared mid-run, the next instance can resume from those files plus `git log`. Keep them up to date.
19
- - Best results come from `xhigh` effort. If you are running on lower effort and notice shallow reasoning during phase decisions, escalate.
13
+ Long-horizon agentic work. Context auto-compacts — do not stop early on token-budget concerns. All durable state lives in `.devlyn/pipeline.state.json` (control plane: pointers, criteria, verdicts) plus `<phase>.findings.jsonl` + `<phase>.log.md` for phases that emit findings. `state.json` is the **single authoritative verdict source** — branch on `phases.<name>.verdict` directly, never parse artifact files. At PHASE 5, the run's `.devlyn/*` artifacts are **archived** to `.devlyn/runs/<run_id>/` (last 10 kept, best-effort). Schemas: `references/pipeline-state.md`, `references/findings-schema.md`. Best results with `xhigh` reasoning.
20
14
  </orchestrator_context>
21
15
 
22
16
  <autonomy_contract>
23
- This pipeline runs hands-free. The user launches it to walk away and come back to finished work, so the quality of this run is measured by how far it gets without human intervention. Apply these behaviors throughout every phase:
17
+ This pipeline runs hands-free. Measured by how far it gets without human intervention.
24
18
 
25
- 1. **Make decisions autonomously and log them in the final report.** When you would otherwise ask the user something ("Should I commit this?", "Ready to proceed?", "Which approach?"), pick the safe default, proceed, and record the decision in PHASE 8's report so the user can review it at the end.
26
- 2. **Run only the phases defined below, in the order given.** Doc updates, roadmap edits, changelog entries, and planning-doc changes belong in PHASE 7 (Docs). Resist inserting them earlier as freelance pre-work.
27
- 3. **Delegate all file changes to spawned subagents.** As the orchestrator, your actions are: parse input, spawn phase agents, read handoff files (`.devlyn/*.md`), run `git` commands, branch on verdicts, and emit the final report.
28
- 4. **Continue through the pipeline by default.** Stop only for: (a) a subagent reporting an unrecoverable failure, (b) PHASE 1 producing zero code changes, (c) `max-rounds` reached — in which case continue to PHASE 3 with a warning rather than halting. Every other situation means move on to the next phase.
29
- 5. **Treat questions as a signal to act instead.** If you notice yourself drafting a question to the user mid-pipeline, convert it into a decision + log entry and spawn the next phase.
19
+ 1. **Never prompt the user mid-pipeline.** When you'd otherwise ask, pick the safe default, proceed, and log it in the final report.
20
+ 2. **Codex availability**: on `--engine auto`/`codex`, call `mcp__codex-cli__ping` at the start. On failure, silently fall back to `--engine claude` and log `engine downgraded: codex-ping failed` in the final report. Do NOT present a menu. Do NOT abort.
21
+ 3. **Run only the phases defined below, in order.** Doc updates belong in PHASE 4 (DOCS). Don't insert them earlier.
22
+ 4. **Delegate all file changes to spawned subagents.** Orchestrator actions: parse input, spawn phase agents, read handoff files, run `git`, branch on verdicts, emit report, archive.
23
+ 5. **Continue by default.** Stop only for: (a) unrecoverable subagent failure, (b) PHASE 1 producing zero code changes, (c) build-gate / browser fix-loop exhausting `max_rounds` (halt FINAL REPORT). EVAL/CRITIC exhaustion proceeds with warning never halts.
30
24
  </autonomy_contract>
31
25
 
32
- <engine_routing_convention>
33
- Every phase in this pipeline routes its work to the optimal model per `references/engine-routing.md`. The convention is the same everywhere:
26
+ <harness_principles>
27
+ Goal-first. Verify state, source integrity, diff base, artifact contracts. Prefer deletion or reuse over new machinery. Change only files the task requires. Each phase optimizes for its declared success criteria, not a checklist. Fix root causes only — no `any`, `@ts-ignore`, silent catches, hardcoded values. Label hypotheses explicitly; back claims with file:line evidence.
28
+ </harness_principles>
34
29
 
35
- - The phase prompt body below is **engine-agnostic** — same instructions whether Codex or Claude executes it.
36
- - For phases routed to **Codex** (per the routing table), call `mcp__codex-cli__codex` per the patterns in `engine-routing.md` (How to Spawn a Codex BUILD/FIX Agent / How to Spawn a Codex Role / How to Spawn a Dual Role).
37
- - For phases routed to **Claude**, spawn an Agent subagent with `mode: "bypassPermissions"` and pass the prompt body verbatim.
38
- - `--engine claude` forces all phases to Claude. `--engine codex` forces implementation/analysis to Codex (Claude still handles orchestration and Chrome MCP). `--engine auto` (default) uses the routing table per phase.
30
+ <engine_routing_convention>
31
+ Every phase routes to the optimal model per `references/engine-routing.md`:
39
32
 
40
- Phase-level "Engine routing" notes below are short reminders only — `engine-routing.md` is the single source of truth.
33
+ - Phase prompt bodies (in `references/phases/`) are engine-agnostic.
34
+ - Phases routed to **Codex**: call `mcp__codex-cli__codex` per spawn patterns in `engine-routing.md`.
35
+ - Phases routed to **Claude**: spawn an `Agent` subagent with `mode: "bypassPermissions"`, passing the phase body verbatim.
36
+ - **Dual** (CRITIC security sub-pass on `--engine auto`): spawn both in parallel; orchestrator merges findings.
37
+ - `--engine claude` forces all phases to Claude. `--engine codex` forces implementation to Codex, orchestration/Chrome MCP stays Claude. `--engine auto` (default) uses the routing table.
41
38
  </engine_routing_convention>
42
39
 
43
- ## PHASE 0: PARSE INPUT
44
-
45
- 1. Extract the task/issue description from `<pipeline_config>`.
46
- 2. Determine optional flags from the input (defaults in parentheses):
47
- - `--max-rounds N` (4) — max evaluate-fix loops before stopping with a report
48
- - `--skip-review` (false) — skip team-review phase
49
- - `--security-review` (auto) — run dedicated security audit. Auto-detects: runs when changes touch auth, secrets, user data, API endpoints, env/config, or crypto. Force with `--security-review always` or skip with `--security-review skip`
50
- - `--skip-clean` (false) — skip clean phase
51
- - `--skip-browser` (false) — skip browser validation phase (auto-skipped for non-web changes)
52
- - `--skip-docs` (false) — skip update-docs phase
53
- - `--skip-build-gate` (false) — skip the deterministic build gate (Phase 1.4). Not recommended — the build gate is the primary defense against "tests pass locally, breaks in CI/Docker/production" class of bugs.
54
- - `--build-gate MODE` (auto) — controls build gate behavior. `auto`: detect project type and run appropriate build/typecheck/lint commands; if Dockerfile(s) are present, Docker builds are included automatically. `strict`: auto + treat warnings as errors. `no-docker`: auto but skip Docker builds even if Dockerfiles exist (for faster iteration). `skip`: same as --skip-build-gate.
55
- - `--engine MODE` (auto) — controls which model handles each pipeline phase and team role. Modes:
56
- - `auto` (default): each phase and team role routes to the optimal model based on benchmark data. Requires Codex MCP server. Codex handles BUILD/FIX (SWE-bench Pro lead) and several team roles; Claude handles EVALUATE, CHALLENGE, BROWSER, and orchestration — creating a GAN-like dynamic where the builder and critic are always different models.
57
- - `codex`: Codex handles implementation/analysis phases, Claude handles orchestration, evaluation, and Chrome MCP.
58
- - `claude`: all phases use Claude subagents. No Codex calls.
59
-
60
- Flags can be passed naturally: `/devlyn:auto-resolve fix the auth bug --max-rounds 3 --skip-docs`
61
- Engine examples: `--engine auto`, `--engine codex`, `--engine claude`
62
- If no flags are present, use defaults. The default engine is `auto` — if the user does not pass `--engine`, treat it as `--engine auto`.
63
-
64
- **Consolidated flag**: `--with-codex` (and its variants `evaluate`/`review`/`both`) was rolled into the smarter `--engine auto` default. If the user passes it, inform them once and proceed with `--engine auto`: "Note: `--with-codex` was consolidated into `--engine auto` (default), which provides broader Codex coverage — Codex now handles BUILD, FIX, and several team roles automatically. No flag needed. Continuing with `--engine auto`."
65
-
66
- 3. **Engine pre-flight** (runs unless `--engine claude` was explicitly passed):
67
- - The default engine is `auto`. If the user did not pass `--engine`, the engine is `auto` — not `claude`.
68
- - Read `references/engine-routing.md` for the full routing table.
69
- - Call `mcp__codex-cli__ping` to verify the Codex MCP server is available. If ping fails, warn the user and offer: [1] Continue with `--engine claude` (fallback), [2] Abort.
70
-
71
- 4. Announce the pipeline plan:
72
- ```
73
- Auto-resolve pipeline starting
74
- Task: [extracted task description]
75
- Engine: [auto / codex / claude]
76
- Phases: Build → Build Gate → [Browser] → Evaluate → [Fix loop if needed] → Simplify → [Review] → Challenge → [Security] → [Clean] → [Docs]
77
- Max evaluation rounds: [N]
78
- ```
79
-
80
- ## PHASE 0.5: SPEC PREFLIGHT (conditional)
81
-
82
- This phase exists because the ideate skill produces specs that are explicitly designed to be auto-resolve's contract — `Requirements` *are* the done-criteria, `Out of Scope` bounds over-building, `Dependencies` gates sequencing. When a run ignores that contract and re-derives everything from the raw task string, 25–40% of BUILD's reasoning is spent re-inventing material the spec already owns. This phase makes the contract load-bearing.
40
+ <post_eval_invariant>
41
+ Once `state.eval_passed_sha` is non-null (PHASE 2 returned PASS or PASS_WITH_ISSUES), the post-EVAL phases (CRITIC, DOCS) run **findings-only / doc-only** — they never write code. DOCS is the only phase allowed to commit after EVAL, and only for doc files.
83
42
 
84
- Scan the task description from `<pipeline_config>` for a path matching the regex `docs/roadmap/phase-\d+/[^\s"'`)]+\.md`. If no match, skip this entire phase (non-spec tasks fall back to BUILD's open-ended discovery that mode is still supported).
43
+ **Orchestrator enforcement (per-phase, NOT cumulative)**: before each post-EVAL phase, capture `state.phases.<phase>.pre_sha = git rev-parse HEAD`. After the subagent completes, run `git diff --name-only <pre_sha> -- ':!.devlyn/**'`:
44
+ - CRITIC (findings-only) → any diff → `git reset --hard <pre_sha>`, emit `rule_id: "invariant.post-eval-code-mutation"` + `severity: HIGH` into `.devlyn/invariant.findings.jsonl`, route to FIX LOOP with `triggered_by: "critic"`.
45
+ - DOCS → check against allowlist; non-allowlisted paths trigger the revert-and-find flow.
85
46
 
86
- If a match is found:
47
+ Per-phase (not cumulative) baseline is correct because fix-loop commits between one post-EVAL phase and the next are legitimate.
87
48
 
88
- 1. **Read the spec file.** If the file does not exist, stop with a `BLOCKED` verdict in the final report do not proceed to BUILD with a missing spec. The task description is lying and recovering from that silently is worse than halting.
49
+ Doc-file allowlist (DOCS): `*.md`, `.mdx`, files under `docs/`, `README*`, `CHANGELOG*`, `CLAUDE.md`, frontmatter in spec files under `docs/roadmap/phase-*/`. Any other path triggers revert-and-find.
50
+ </post_eval_invariant>
89
51
 
90
- 2. **Verify internal dependencies.** For each entry under the spec's `## Dependencies` → `Internal` list (e.g., `1.1 User Auth`), locate the matching spec file at `docs/roadmap/phase-*/[id]-*.md` and read its frontmatter `status` field. If any internal dependency does not have `status: done`, stop with a `BLOCKED` verdict listing the unmet deps. Implementing out of sequence wastes the whole pipeline and produces code that fails at the first integration point.
52
+ <perf_opt_in>
53
+ Optional: pass `--perf` to record per-phase `{wall_ms, tokens, engine, round, triggered_by}` into `state.perf.per_phase` and totals at PHASE 5. Off by default. Harness efficiency claims can be measured when needed; mandatory meta-measurement was retired in v3.4.
54
+ </perf_opt_in>
91
55
 
92
- 3. **Write `.devlyn/SPEC-CONTEXT.md`** so downstream subagents read spec-owned content from a single canonical place without re-parsing the spec file. Copy these spec sections verbatim (do not paraphrase or compress — they are the contract):
56
+ ## PHASE 0: PARSE + PREFLIGHT + ROUTE
93
57
 
94
- ```
95
- ---
96
- id: [from frontmatter]
97
- complexity: [from frontmatter]
98
- priority: [from frontmatter]
99
- depends-on: [from frontmatter]
100
- source-spec: [path to the spec file]
101
- ---
58
+ 1. **Parse flags** from `<pipeline_config>`:
59
+ - `--max-rounds N` (4)
60
+ - `--route MODE` (auto) — per `references/pipeline-routing.md`
61
+ - `--engine MODE` (auto) — per `references/engine-routing.md`
62
+ - `--team` — force team-assembled BUILD even on non-strict routes (default: solo).
63
+ - `--bypass <phase>[,<phase>...]` — skip specific phases. Valid: `build-gate`, `browser`, `critic`, `docs`. Deprecated aliases (`--skip-*`, `--security-review skip`, `--bypass simplify|review|clean|security|challenge`) map to `--bypass critic` where applicable; log `deprecated flag — use --bypass <phase>` once.
64
+ - `--build-gate MODE` (auto) `auto` / `strict` / `no-docker`.
65
+ - `--perf` — opt in to per-phase timing/token accounting.
102
66
 
103
- ## Customer Frame
104
- [verbatim]
67
+ 2. **Engine pre-flight** (unless `--engine claude`): call `mcp__codex-cli__ping`. On failure, silent fallback to `--engine claude`, log `engine downgraded`. Never prompt.
105
68
 
106
- ## Objective
107
- [verbatim]
69
+ 3. **Initialize `pipeline.state.json`** per `references/pipeline-state.md`:
70
+ - `version: "1.2"`, `run_id: "ar-$(date -u +%Y%m%dT%H%M%SZ)-<12-hex>"`, `started_at`, `engine`, `base_ref.{branch, sha}`, `rounds.max_rounds`, `eval_passed_sha: null`, `route.bypasses: [...]`, empty `phases`, `criteria`, `route.selected`.
108
71
 
109
- ## Requirements
110
- [verbatim these become done-criteria in PHASE 1]
72
+ 4. **Spec preflight** (if `<pipeline_config>` contains `docs/roadmap/phase-\d+/[^\s"'`)]+\.md`):
73
+ - Read the spec. Missing `BLOCKED`.
74
+ - Verify internal deps (each entry under `## Dependencies → Internal` resolves to a `status: done` spec). Unmet → `BLOCKED`.
75
+ - Populate `state.source`: `type: "spec"`, `spec_path`, `spec_sha256 = sha256(spec)`, `criteria_anchors: ["spec://requirements", "spec://out-of-scope", "spec://verification", "spec://constraints", "spec://architecture-notes", "spec://dependencies"]`.
76
+ - Populate `state.criteria[]`: one per `- [ ]` in `## Requirements`, `status: pending`.
111
77
 
112
- ## Constraints
113
- [verbatim]
78
+ No spec path found → `source.type: "generated"`, `source.criteria_path: ".devlyn/criteria.generated.md"` (PHASE 1 creates it), `criteria_anchors: ["criteria.generated://requirements", "criteria.generated://out-of-scope", "criteria.generated://verification"]`, `criteria: []`.
114
79
 
115
- ## Out of Scope
116
- [verbatim — honored explicitly by BUILD in Phase D]
80
+ 5. **Compute Stage A route** per `references/pipeline-routing.md#stage-a`. Write to `state.route.{selected, user_override, stage_a}`.
117
81
 
118
- ## Architecture Notes
119
- [verbatim, or "(none)" if absent]
120
-
121
- ## Dependencies
122
- [verbatim]
123
-
124
- ## Verification
125
- [verbatim]
126
- ```
127
-
128
- 4. **Announce the preflight outcome.** One line: `Spec preflight: [spec path] — complexity [low/medium/high], [N] internal deps verified done, proceeding.` This appears in the final report under the Build row.
129
-
130
- Downstream phases detect `.devlyn/SPEC-CONTEXT.md` and prefer its content over re-derivation. If it is absent, they use their current open-ended behavior.
82
+ 6. **Announce** (single line):
83
+ ```
84
+ Auto-resolve starting — run <run_id> — task: <desc>
85
+ Engine: <engine>, Route: <selected> (<stage_a_reasons>), Bypasses: <bypasses|none>, Max rounds: <N>
86
+ ```
131
87
 
132
88
  ## PHASE 1: BUILD
133
89
 
134
- **Engine**: BUILD row of the routing table — Codex on `auto`/`codex`, Claude on `claude`. Per `<engine_routing_convention>` above. Subagents do not have access to skills, so the prompt below includes everything they need inline.
135
-
136
- Agent prompt — pass this to the spawned executor:
137
-
138
- Investigate and implement the following task. Work through these phases in order:
90
+ **Engine**: BUILD row. Spawn per `<engine_routing_convention>`. Prompt body: **`references/phases/phase-1-build.md`** (verbatim) + task description.
139
91
 
140
- **Phase A Understand the task**: If `.devlyn/SPEC-CONTEXT.md` exists, read it first. The spec has already decided the task shape — use its `Objective`, `Constraints`, `Architecture Notes`, `Dependencies`, and `complexity` as the exploration boundary. Do not re-classify the task type open-endedly; the spec already bounds the problem. Read only the files the spec implicates (Architecture Notes + Dependencies + any existing files touched by referenced patterns), then move on.
141
-
142
- If no spec context file exists, read the raw task description and classify the task type:
143
- - **Bug fix**: trace from symptom to root cause. Read error logs and affected code paths.
144
- - **Feature**: explore the codebase to find existing patterns, integration points, and relevant modules.
145
- - **Refactor/Chore**: understand current implementation, identify what needs to change and why.
146
- - **UI/UX**: review existing components, design system, and user flows.
147
- Read relevant files in parallel. Build a clear picture of what exists and what needs to change.
148
-
149
- **Phase B — Define done criteria**: Before writing any code, create `.devlyn/done-criteria.md`.
150
-
151
- First check whether `.devlyn/SPEC-CONTEXT.md` exists (produced by PHASE 0.5 when this run implements an ideate-produced spec). If it does, the spec is the contract — copy its `## Requirements` section verbatim into `done-criteria.md` as the primary done-criteria list, copy its `## Out of Scope` section as an `## Out of Scope` section in done-criteria.md, and copy its `## Verification` section as a `## Verification Method` section. Do not paraphrase, compress, or re-derive these — the ideate skill's CHALLENGE rubric already validated them, and weakening them here silently undoes that work. You may ADD criteria the spec obviously missed (e.g., if Requirements mention an API but omit an obvious error state) but never REMOVE or reword existing ones.
152
-
153
- If `.devlyn/SPEC-CONTEXT.md` does not exist, synthesize done-criteria from the raw task description. Each criterion must be verifiable (a test can assert it or a human can observe it in under 30 seconds), specific (not vague like "handles errors correctly"), and scoped to this task. Include an "Out of Scope" section and a "Verification Method" section.
154
-
155
- This file is required — downstream evaluation depends on it.
156
-
157
- **Phase C — Assemble a team (complexity-gated)**: Check `.devlyn/SPEC-CONTEXT.md` frontmatter for `complexity`.
158
-
159
- If `complexity: low` AND the spec does not touch security/auth/API/data/UI risk areas (check by greping the spec for keywords: `auth`, `login`, `session`, `token`, `secret`, `password`, `crypto`, `api`, `env`, `permission`, `access`, `database`, `migration`, `payment`), skip TeamCreate entirely and implement directly — the multi-perspective team exists to catch ambiguity that low-complexity specs have already resolved.
160
-
161
- Otherwise (complexity medium or high, risk areas present, or no spec context), use TeamCreate to create a team. Select teammates based on task type:
162
- - Bug fix: root-cause-analyst + test-engineer (+ security-auditor, performance-engineer as needed)
163
- - Feature: implementation-planner + test-engineer (+ ux-designer, architecture-reviewer, api-designer as needed)
164
- - Refactor: architecture-reviewer + test-engineer
165
- - UI/UX: product-designer + ux-designer + ui-designer (+ accessibility-auditor as needed)
166
- Each teammate investigates from their perspective and sends findings back. Per-role engine routing follows the team-resolve table in `references/engine-routing.md`; Dual roles run both models in parallel.
167
-
168
- **Phase D — Synthesize and implement**: After all teammates report, compile findings into a unified plan. Implement the solution — no workarounds, no hardcoded values, no silent error swallowing. For bugs: write a failing test first, then fix. For features: implement following existing patterns, then write tests. For refactors: ensure tests pass before and after.
169
-
170
- **Phase E — Update done criteria**: Mark each criterion in `.devlyn/done-criteria.md` as satisfied. Run the full test suite.
171
-
172
- **Phase F — Cleanup**: Shut down all teammates and delete the team.
173
-
174
- The task is: [paste the task description here]
92
+ **Team assembly rule** (simplified from v3.2): BUILD spawns as **team** ONLY when `--team` flag passed OR `state.route.selected == "strict"`. Otherwise solo. Keyword-match auto-trigger removed Claude/Codex base SWE capability is the default.
175
93
 
176
94
  **After the agent completes**:
177
- 1. Verify `.devlyn/done-criteria.md` exists if missing, create a basic one from the agent's output summary
178
- 2. Run `git diff --stat` to confirm code was actually changed
179
- 3. If no changes were made, report failure and stop
180
- 4. **Checkpoint**: Run `git add -A && git commit -m "chore(pipeline): phase 1 — build complete"` to create a rollback point
95
+ 1. Verify `criteria[]` has ≥1 entry with `status != "pending"`. If not, re-spawn with reminder.
96
+ 2. `git diff --stat` if no changes, halt with failure.
97
+ 3. Checkpoint: `git add -A && git commit -m "chore(pipeline): phase 1 — build complete"`.
181
98
 
182
99
  ## PHASE 1.4: BUILD GATE
183
100
 
184
- Skip if `--skip-build-gate` or `--build-gate skip` was set.
185
-
186
- This phase runs the project's real build, typecheck, and lint commands — the same ones CI, Docker, and production environments will run. It catches the entire class of bugs that LLM-based evaluation and test suites cannot: type errors in un-tested files, cross-package type drift in monorepos, lint violations, missing production dependencies, and Dockerfile copy mismatches.
187
-
188
- This is deterministic — if the compiler says no, the pipeline stops. No LLM judgment involved.
189
-
190
- Spawn a subagent using the Agent tool with `mode: "bypassPermissions"`.
101
+ Skip if `build-gate` in `state.route.bypasses`. Deterministic — same commands CI/Docker/production run.
191
102
 
192
- Agent prompt pass this to the Agent tool:
193
-
194
- You are the build gate agent. Read `references/build-gate.md` from the auto-resolve skill directory for the full project-type detection matrix and execution rules.
195
-
196
- Your job: detect every project type in this repo, run their build/typecheck/lint commands, and report results. You do NOT reason about code quality — you run commands and faithfully report what they output.
197
-
198
- 1. Read the detection matrix in `references/build-gate.md`
199
- 2. Scan the repo to detect all matching project types (a monorepo may match several)
200
- 3. Detect the package manager (npm/pnpm/yarn/bun) per the rules in the reference file
201
- 4. Run all gate commands. Sequential within a project type, parallel across unrelated types.
202
- 5. If `--build-gate strict` is set, apply strict-mode flags per the reference file
203
- 6. Run Dockerfile builds if Dockerfiles are detected, UNLESS `--build-gate no-docker` is set (see reference file)
204
- 7. Write results to `.devlyn/BUILD-GATE.md` following the output format in the reference file
205
-
206
- For failures: include the FULL error output (not truncated) and extract root file:line references with concrete fix guidance so the fix agent knows exactly where to look.
207
-
208
- **After the agent completes**:
209
- 1. Read `.devlyn/BUILD-GATE.md`
210
- 2. Extract verdict
211
- 3. Branch:
212
- - `PASS` → continue to PHASE 1.5
213
- - `FAIL` → go to PHASE 1.4-fix (build gate fix loop)
214
-
215
- ## PHASE 1.4-fix: BUILD GATE FIX LOOP
216
-
217
- Triggered only when PHASE 1.4 returns FAIL.
218
-
219
- Track a round counter. The build-gate fix loop and the main evaluate fix loop share **one global round counter** capped at `max-rounds` — increments from this loop and from PHASE 2.5 both count against the same total. If `round >= max-rounds`, stop with a clear failure report and do not continue to evaluate/browser/etc. Code that doesn't build cannot be meaningfully evaluated or tested.
220
-
221
- **Engine**: FIX LOOP row of the routing table.
222
-
223
- Agent prompt — pass this to the spawned executor:
224
-
225
- Read `.devlyn/BUILD-GATE.md` — it contains deterministic build/typecheck/lint failures from real compiler output. These are not opinions; the compiler rejected this code. Fix every listed failure at the root cause level.
226
-
227
- For each failure:
228
- 1. Read the referenced file:line and enough surrounding context to understand the error
229
- 2. For type errors: check BOTH sides of the type contract — the consumer AND the type definition. The fix may belong to either side. Do NOT suppress errors with `any`, `@ts-ignore`, `as unknown as`, `// eslint-disable`, or equivalent escape hatches.
230
- 3. For lint errors: fix the underlying issue, do not disable the rule.
231
- 4. For missing module/dependency errors: investigate the cause — it may be a missing dep in package.json, a typo in the import path, or a tsconfig paths misconfiguration.
232
- 5. After fixing, do NOT re-run the build yourself. The orchestrator re-runs PHASE 1.4.
103
+ Spawn Claude `Agent` (`mode: "bypassPermissions"`): "Read `references/build-gate.md` (detection matrix, commands, package manager, monorepo, strict, Docker) and `references/findings-schema.md`. Run all matched gates. Apply strict flags if `--build-gate strict` OR `state.route.selected == "strict"`. Run Docker unless `--build-gate no-docker`. Emit `.devlyn/build_gate.findings.jsonl` + `.devlyn/build_gate.log.md`; update `state.phases.build_gate`."
233
104
 
234
105
  **After the agent completes**:
235
- 1. **Checkpoint**: `git add -A && git commit -m "chore(pipeline): build gate fix round [N]"`
236
- 2. Increment the global round counter (shared with PHASE 2.5)
237
- 3. Go back to PHASE 1.4 (re-run the gate)
106
+ 1. Read `state.phases.build_gate.verdict`.
107
+ 2. **Stage B LITE** (only if `verdict == "PASS"` AND `state.route.user_override == false`): apply the single escalation rule from `references/pipeline-routing.md#stage-b-lite`. If it fires, write `state.route.stage_b.{at, escalated_from, reasons}`.
108
+ 3. Branch: `PASS` PHASE 1.5; `FAIL` PHASE 2.5 with `triggered_by: "build_gate"`.
238
109
 
239
110
  ## PHASE 1.5: BROWSER VALIDATE (conditional)
240
111
 
241
- Skip if `--skip-browser` was set.
242
-
243
- 1. **Check relevance**: Run `git diff --name-only` and check for web-relevant files (`*.tsx`, `*.jsx`, `*.vue`, `*.svelte`, `*.css`, `*.html`, `page.*`, `layout.*`, `route.*`). If none found, skip and note "Browser validation skipped — no web changes detected."
244
-
245
- 2. **Run validation**: Spawn a subagent using the Agent tool with `mode: "bypassPermissions"`.
112
+ Skip if `browser` in `state.route.bypasses`. Skip if `git diff --name-only <state.base_ref.sha>` has no `*.tsx`, `*.jsx`, `*.vue`, `*.svelte`, `*.css`, `*.html`, `page.*`, `layout.*`, `route.*` matches.
246
113
 
247
- Agent prompt pass this to the Agent tool:
248
-
249
- You are a browser validation agent. Read the skill instructions at `.claude/skills/devlyn:browser-validate/SKILL.md` and follow the full workflow to validate this web application. The dev server should be started, tested, and left running (pass `--keep-server` internally) — the pipeline will clean it up later. Write your findings to `.devlyn/BROWSER-RESULTS.md`.
114
+ Spawn Claude `Agent` (`mode: "bypassPermissions"`): "Read `.claude/skills/devlyn:browser-validate/SKILL.md` (tiered Chrome MCP → Playwright → curl) and `references/findings-schema.md`. Start dev server, test the implemented feature end-to-end against `pipeline.state.json:criteria[]`, leave server running (`--keep-server`). Emit `.devlyn/browser_validate.findings.jsonl` + `.devlyn/browser_validate.log.md`; update `state.phases.browser_validate`."
250
115
 
251
116
  **After the agent completes**:
252
- 1. Read `.devlyn/BROWSER-RESULTS.md`
253
- 2. Extract the verdict
254
- 3. **Validate the verdict is real**: If the verdict says "code-level pass" or indicates no actual browser interaction occurred (no screenshots taken, no pages navigated, no DOM inspected), the validation did NOT happen. Treat this as if no browser validation ran — re-run PHASE 1.5 with `--tier 2` to force Playwright, or `--tier 3` for HTTP smoke. A "PARTIALLY VERIFIED" based on reading source code is not browser validation.
255
- 4. Branch on verdict:
256
- - `PASS` → continue to PHASE 2
257
- - `PASS WITH ISSUES` → continue to PHASE 2 (evaluator reads browser results as extra context)
258
- - `PARTIALLY VERIFIED` → continue to PHASE 2, but flag to the evaluator that browser coverage was incomplete — unverified features should be weighted more heavily. This verdict is only valid when features were actually tested in a browser and some couldn't be verified due to environment limitations (missing API keys, external services). It is NOT valid as a substitute for "browser tools didn't work."
259
- - `NEEDS WORK` → features don't work in the browser. Go to PHASE 2.5 fix loop. Fix agent reads `.devlyn/BROWSER-RESULTS.md` for which criterion failed, at what step, with what error. After fixing, re-run PHASE 1.5 to verify the fix before proceeding to Evaluate.
260
- - `BLOCKED` → app doesn't render. Go to PHASE 2.5 fix loop. After fixing, re-run PHASE 1.5.
117
+ 1. **Sanity check**: if verdict is `PASS`/`PASS_WITH_ISSUES` but log shows zero screenshots AND zero navigations, treat as unverified — re-run at `--tier 2`/`3`. Code-level verdict is not browser validation.
118
+ 2. Branch: `PASS`/`PASS_WITH_ISSUES`/`PARTIALLY_VERIFIED` → PHASE 2; `NEEDS_WORK`/`BLOCKED` → PHASE 2.5 with `triggered_by: "browser_validate"`.
261
119
 
262
120
  ## PHASE 2: EVALUATE
263
121
 
264
- **Engine**: EVALUATE row of the routing table — Claude on every engine. When `--engine auto`, Codex built the code, so Claude evaluating Codex's work is the GAN dynamic by default; no separate Codex evaluation pass is needed.
265
-
266
- Spawn a subagent using the Agent tool with `mode: "bypassPermissions"`. Include all evaluation instructions inline (subagents do not have access to skills).
267
-
268
- Agent prompt — pass this to the spawned executor:
269
-
270
- You are an independent evaluator. Your job is to grade work produced by another agent against a specific rubric, not to praise it.
271
-
272
- <investigate_before_answering>
273
- Never claim a file:line or assert a behavior you have not opened and read. The done-criteria file is the rubric — read it first. Then read every changed/new file in full before marking anything VERIFIED or FAILED. Findings without a real file:line behind them are speculation; exclude them.
274
- </investigate_before_answering>
275
-
276
- <coverage_over_filtering>
277
- Your goal is coverage at this stage, not severity filtering. Report every issue you find — uncertain ones, low-severity ones, all of them. The fix loop and the orchestrator's verdict logic do the filtering downstream. Each finding includes its severity and your confidence so the downstream layers can rank them; your job is to surface them, not pre-decide which ones matter.
278
-
279
- This matters because under-reporting is the asymmetric cost: a missed bug ships broken code, a flagged non-issue costs a few minutes of review.
280
- </coverage_over_filtering>
281
-
282
- **Step 1 — Read the done criteria**: Read `.devlyn/done-criteria.md`. This is your primary grading rubric. Every criterion must be verified with evidence.
283
-
284
- **Step 2 — Discover changes**: Run `git diff HEAD~1` and `git status` to see what changed. Read all changed/new files in parallel.
285
-
286
- **Step 3 — Evaluate**: For each changed file, check:
287
- - Correctness: logic errors, silent failures, null access, incorrect API contracts
288
- - Architecture: pattern violations, duplication, missing integration
289
- - Security (if auth/secrets/user-data touched): injection, hardcoded credentials, missing validation
290
- - Frontend (if UI changed): missing error/loading/empty states, React anti-patterns, server/client boundaries
291
- - Test coverage: untested modules, missing edge cases
292
-
293
- **Step 4 — Grade against done criteria**: For each criterion in done-criteria.md, mark VERIFIED (with evidence) or FAILED (with file:line and what's wrong).
294
-
295
- **Step 5 — Write findings**: Write `.devlyn/EVAL-FINDINGS.md` with this exact structure:
296
-
297
- ```
298
- # Evaluation Findings
299
- ## Verdict: [PASS / PASS WITH ISSUES / NEEDS WORK / BLOCKED]
300
- ## Done Criteria Results
301
- - [x] criterion — VERIFIED: evidence
302
- - [ ] criterion — FAILED: what's wrong, file:line
303
- ## Findings Requiring Action
304
- ### CRITICAL
305
- - `file:line` — description — Confidence: high/med/low — Fix: suggested approach
306
- ### HIGH
307
- - `file:line` — description — Confidence: high/med/low — Fix: suggested approach
308
- ### MEDIUM / LOW
309
- - `file:line` — description — Confidence: high/med/low — Fix: suggested approach
310
- ## Cross-Cutting Patterns
311
- - pattern description
312
- ```
313
-
314
- Verdict rules:
315
- - `BLOCKED` — any CRITICAL issues
316
- - `NEEDS WORK` — HIGH or MEDIUM issues
317
- - `PASS WITH ISSUES` — only LOW cosmetic notes
318
- - `PASS` — clean
319
-
320
- Findings labeled "pre-existing" or "out of scope" still count if they relate to the done criteria. The goal is working software, not blame attribution.
321
-
322
- Calibration examples:
323
- - A catch block that logs but doesn't surface the error to the user → HIGH (not MEDIUM). Logging is not error handling.
324
- - A `let` that could be `const` → LOW. Linters catch this.
325
- - "The error handling is generally quite good" is not a finding. Count the instances and name the files. "3 of 7 async ops have error states. 4 are missing: file:line, file:line…"
326
-
327
- Do not delete `.devlyn/done-criteria.md` or `.devlyn/EVAL-FINDINGS.md` — the orchestrator needs them.
328
-
329
- **After the agent completes**:
330
- 1. Read `.devlyn/EVAL-FINDINGS.md`
331
- 2. Extract the verdict
332
- 3. Branch on verdict:
333
- - `PASS` → skip to PHASE 3
334
- - `PASS WITH ISSUES` → go to PHASE 2.5 (fix loop) — LOW-only issues are still issues; fix them
335
- - `NEEDS WORK` → go to PHASE 2.5 (fix loop)
336
- - `BLOCKED` → go to PHASE 2.5 (fix loop)
337
- 4. If `.devlyn/EVAL-FINDINGS.md` was not created, treat as NEEDS WORK and log a warning — absence of evidence is not evidence of absence
338
-
339
- ## PHASE 2.5: FIX LOOP (conditional)
340
-
341
- Track the current round number. If `round >= max-rounds`, stop the loop and proceed to PHASE 3 with a warning that unresolved findings remain.
342
-
343
- **Engine**: FIX LOOP row of the routing table. Use a fresh Codex call each round (no `sessionId` reuse — sandbox/fullAuto only apply on the first call of a session).
344
-
345
- Agent prompt — pass this to the spawned executor:
346
-
347
- Read every findings file present in `.devlyn/`:
348
- - `.devlyn/EVAL-FINDINGS.md` — issues from the independent evaluator (PHASE 2)
349
- - `.devlyn/BROWSER-RESULTS.md` — issues from browser validation (PHASE 1.5), if present and the verdict is `NEEDS WORK` or `BLOCKED`
350
-
351
- Fix every finding regardless of severity (CRITICAL, HIGH, MEDIUM, and LOW). The pipeline loops until the relevant verdict returns PASS — there is no "shippable with issues" shortcut.
352
-
353
- The original done criteria are in `.devlyn/done-criteria.md` — your fixes must still satisfy those criteria. Do not delete or weaken criteria to make them pass.
354
-
355
- For each finding: read the referenced file:line (or browser step / console error), understand the issue, implement the fix. No workarounds — fix the actual root cause. Run tests after fixing. Update `.devlyn/done-criteria.md` to mark fixed items.
122
+ **Engine**: EVAL row always Claude. Prompt body: **`references/phases/phase-2-evaluate.md`**.
356
123
 
357
124
  **After the agent completes**:
358
- 1. **Checkpoint**: Run `git add -A && git commit -m "chore(pipeline): fix round [N] complete"` to preserve the fix
359
- 2. Increment the global round counter (shared with PHASE 1.4-fix)
360
- 3. Re-run the phase that triggered the fix:
361
- - If invoked from PHASE 2 (eval failure) → go back to PHASE 2 to re-evaluate
362
- - If invoked from PHASE 1.5 (browser failure) → go back to PHASE 1.5 to re-validate the browser, then proceed to PHASE 2 only if browser passes
363
-
364
- ## PHASE 3: SIMPLIFY
365
-
366
- Spawn a subagent using the Agent tool with `mode: "bypassPermissions"` for a quick cleanup pass.
367
-
368
- Agent prompt — pass this to the Agent tool:
369
-
370
- Review the recently changed files (use `git diff HEAD~1` to see what changed). Look for: code that could reuse existing utilities instead of reimplementing, quality issues (unclear naming, unnecessary complexity), and efficiency improvements (redundant operations, missing early returns). Fix any issues found. Keep changes minimal — this is a polish pass, not a rewrite.
371
-
372
- **After the agent completes**:
373
- 1. **Checkpoint**: Run `git add -A && git commit -m "chore(pipeline): simplify pass complete"` if there are changes
374
-
375
- ## PHASE 4: REVIEW (skippable)
376
-
377
- Skip if `--skip-review` was set.
378
-
379
- **Engine**: REVIEW (team) — per-role routing per the team-review table in `references/engine-routing.md`. Dual roles run both models in parallel and merge findings.
380
-
381
- Spawn a subagent using the Agent tool with `mode: "bypassPermissions"`.
382
-
383
- Agent prompt — pass this to the spawned executor:
384
-
385
- Review all recent changes in this codebase (use `git diff main` and `git status` to determine scope). Assemble a review team using TeamCreate with specialized reviewers: security reviewer, quality reviewer, test analyst. Add UX reviewer, performance reviewer, or API reviewer based on the changes. Per-role engine routing follows the team-review table in `references/engine-routing.md`; Dual roles run both models in parallel and merge findings.
386
-
387
- Each reviewer reports findings with file:line evidence grouped by severity (CRITICAL, HIGH, MEDIUM, LOW) and a confidence level. After all reviewers report, synthesize findings, deduplicate, and fix any CRITICAL issues directly. For HIGH issues, fix if straightforward.
388
-
389
- Clean up the team after completion.
390
-
391
- **After the review phase completes**:
392
- 1. If CRITICAL issues remain unfixed, log a warning in the final report
393
- 2. **Checkpoint**: Run `git add -A && git commit -m "chore(pipeline): review fixes complete"` if there are changes
394
-
395
- ## PHASE 4.5: CHALLENGE
396
-
397
- Every prior phase used checklists, done-criteria, or structured categories. This phase is deliberately different — it's a fresh pair of eyes with no checklist, no prior context, and a skeptical mandate. The subagent hasn't seen the done-criteria, the eval findings, or the review results. It reads the raw diff cold and asks: "would I mass-ship this?"
398
-
399
- This is what catches the things structured reviews miss — subtle logic that technically works but isn't the right approach, assumptions nobody questioned, patterns that are fine but not best-practice, and integration seams that look correct in isolation but feel wrong when you read the whole changeset.
400
-
401
- **Engine**: CHALLENGE row — Claude on every engine. The diff was likely produced by Codex on `--engine auto`; Claude reading it cold preserves the cross-model dynamic.
402
-
403
- Spawn a subagent using the Agent tool with `mode: "bypassPermissions"`.
404
-
405
- Agent prompt — pass this to the spawned executor:
406
-
407
- You are a senior engineer doing a final skeptical review before this code ships to production. You have not seen any prior reviews, test results, or design docs — read the code cold.
408
-
409
- <investigate_before_answering>
410
- Anchor every finding in code you have actually opened. Run `git diff main` for the change surface, then read each changed file in full (not just the hunks — surrounding context matters). Findings without a real file:line and a quote from the code are speculation; exclude them.
411
- </investigate_before_answering>
412
-
413
- Your job is not to check boxes. Your job is to find the things that would make a staff engineer say "hold on, let's talk about this before we ship." Think about:
414
-
415
- - Would this approach survive a 10x traffic spike? A midnight oncall page? A junior dev maintaining it 6 months from now?
416
- - Are there assumptions baked in that nobody stated out loud? Hardcoded limits, implicit ordering, missing edge cases in business logic?
417
- - Is the error handling actually helpful, or does it just prevent crashes while leaving the user confused?
418
- - Are there simpler, more idiomatic ways to do what this code does? Not "clever" alternatives — genuinely better approaches?
419
- - Would you confidently approve this PR, or would you leave comments?
420
-
421
- Be direct and concrete. Do not open with praise. Every finding must include `file:line` and a concrete fix — not "consider improving" but "change X to Y because Z."
422
-
423
- Write `.devlyn/CHALLENGE-FINDINGS.md`:
424
-
425
- ```
426
- # Challenge Findings
427
- ## Verdict: [PASS / NEEDS WORK]
428
- ## Findings
429
- ### [severity: CRITICAL / HIGH / MEDIUM]
430
- - `file:line` — what's wrong — Fix: concrete change
431
- ```
432
-
433
- <examples>
434
- <example index="1">
435
- GOOD finding (anchored, specific, fixable):
436
- ### CRITICAL
437
- - `src/api/orders/cancel.ts:42` — `await db.transaction(...)` is missing — the read of `order.status` and the write of `order.status = "cancelled"` are not atomic, so two concurrent cancellations both succeed and the inventory hook fires twice. Fix: wrap the read+write in `db.transaction()` and re-check `order.status === "pending"` inside the transaction before the update.
438
- </example>
439
- <example index="2">
440
- BAD finding (vague, unanchored, not actionable):
441
- ### HIGH
442
- - The error handling could be improved. Consider being more defensive throughout.
443
-
444
- Why this is bad: no file:line, no specific failure, no concrete fix. Either delete the finding or replace it with a real one anchored to a specific call site.
445
- </example>
446
- <example index="3">
447
- GOOD finding (idiom / approach issue):
448
- ### MEDIUM
449
- - `src/components/UserList.tsx:18-34` — fetching `/api/users` inside `useEffect` and managing loading/error state by hand re-implements what the project already does with the `useFetch` hook in `src/hooks/useFetch.ts`. Fix: replace the manual `useState`+`useEffect` with `useFetch('/api/users')` so this list inherits retry, cache, and abort handling.
450
- </example>
451
- </examples>
452
-
453
- Verdict: `PASS` only if you would confidently ship this code with your name on it. If you found anything CRITICAL or HIGH, verdict is `NEEDS WORK`.
454
-
455
- **After the agent completes**:
456
- 1. Read `.devlyn/CHALLENGE-FINDINGS.md`
457
- 2. Extract the verdict
125
+ 1. Read `state.phases.evaluate.verdict`.
126
+ 2. **First-time PASS or PASS_WITH_ISSUES** with `state.eval_passed_sha == null` → set `state.eval_passed_sha = git rev-parse HEAD` (activates `<post_eval_invariant>`).
458
127
  3. Branch:
459
- - `PASS` → continue to PHASE 5
460
- - `NEEDS WORK` → spawn a fix subagent with `mode: "bypassPermissions"`:
461
-
462
- Read `.devlyn/CHALLENGE-FINDINGS.md` — it contains findings from a fresh skeptical review. Fix every CRITICAL and HIGH finding at the root cause. For MEDIUM findings, fix if straightforward. After fixing, run the test suite to verify nothing broke.
463
-
464
- After the fix agent completes:
465
- 1. **Checkpoint**: Run `git add -A && git commit -m "chore(pipeline): challenge fixes complete"`
466
- 2. Continue to PHASE 5 (do NOT re-run the challenge — one pass is sufficient to avoid infinite loops)
467
-
468
- ## PHASE 5: SECURITY REVIEW (conditional)
128
+ - `PASS` → PHASE 3 (CRITIC) per route; `fast` → PHASE 5 (FINAL REPORT).
129
+ - `PASS_WITH_ISSUES` → **terminal for this phase** (LOW-only findings do not re-trigger fix loop). Proceed to next phase.
130
+ - `NEEDS_WORK` / `BLOCKED` → PHASE 2.5 with `triggered_by: "evaluate"`.
469
131
 
470
- Determine whether to run this phase:
471
- - If `--security-review always` → run
472
- - If `--security-review skip` → skip
473
- - If `--security-review auto` (default) → auto-detect by scanning changed files for security-sensitive patterns:
474
- - Run `git diff main --name-only` and check for files matching: `*auth*`, `*login*`, `*session*`, `*token*`, `*secret*`, `*crypt*`, `*password*`, `*api*`, `*middleware*`, `*env*`, `*config*`, `*permission*`, `*role*`, `*access*`
475
- - Also run `git diff main` and scan for patterns: `API_KEY`, `SECRET`, `TOKEN`, `PASSWORD`, `PRIVATE_KEY`, `Bearer`, `jwt`, `bcrypt`, `crypto`, `env.`, `process.env`
476
- - If any match → run. If no matches → skip and note "Security review skipped — no security-sensitive changes detected."
132
+ ## PHASE 2.5: UNIFIED FIX LOOP
477
133
 
478
- Spawn a subagent using the Agent tool with `mode: "bypassPermissions"` for a dedicated security audit.
134
+ Single fix loop for every trigger (`build_gate` / `browser_validate` / `evaluate` / `critic`). `state.rounds.global` shared counter.
479
135
 
480
- Agent prompt pass this to the Agent tool:
136
+ **Exhaustion check first**: if `state.rounds.global >= state.rounds.max_rounds`:
137
+ - `build_gate` / `browser_validate` → **halt** → PHASE 5 with exhaustion banner.
138
+ - `evaluate` / `critic` → **proceed_with_warning** → skip to next phase; final report shows banner.
481
139
 
482
- You are a security auditor performing a dedicated security review. This is NOT a general code reviewfocus exclusively on security concerns.
483
-
484
- Examine all recent changes (use `git diff main` to see what changed). For every changed file:
140
+ **Fix-batch packet assembly**: read the trigger's `.findings.jsonl` (plus browser_validate if `triggered_by == "evaluate"` or `"browser_validate"` and browser has open findings see pipeline-routing.md), filter `status == "open"`, write `.devlyn/fix-batch.round-<N>.json`:
141
+ ```json
142
+ {
143
+ "round": <N>, "max_rounds": <N>, "base_ref_sha": "...", "criteria_source": "...",
144
+ "triggered_by": "<trigger>", "findings": [ /* id, rule_id, severity, file, line, message, fix_hint, criterion_ref */ ],
145
+ "failed_criteria": ["<C ids>"], "acceptance": {"build_gate_cmd": "...", "test_cmd": "..."}
146
+ }
147
+ ```
485
148
 
486
- 1. **Input validation**: Trace every user input from entry point to storage/output. Check for: SQL injection, XSS, command injection, path traversal, SSRF.
487
- 2. **Authentication & authorization**: Are new endpoints properly protected? Are auth checks consistent with existing patterns? Any privilege escalation paths?
488
- 3. **Secrets & credentials**: Grep for hardcoded API keys, tokens, passwords, private keys. Check that secrets come from env vars, not source code. Verify .gitignore covers sensitive files.
489
- 4. **Data exposure**: Are error messages leaking internal details? Are logs capturing sensitive data? Are API responses returning more data than needed?
490
- 5. **Dependencies**: If package.json/requirements.txt changed, run the package manager's audit command (npm audit, pip-audit, etc.).
491
- 6. **CSRF/CORS**: For new endpoints with side effects, verify CSRF protection. Check CORS configuration for overly permissive origins.
149
+ **Engine**: FIX LOOP row (Codex on `auto`/`codex`, Claude on `claude`). Fresh Codex call each round (no `sessionId` reuse).
492
150
 
493
- For each finding, provide: severity (CRITICAL/HIGH/MEDIUM), file:line, OWASP category, description, and suggested fix.
151
+ Spawn per `<engine_routing_convention>`. Prompt:
494
152
 
495
- Fix any CRITICAL findings directly. For HIGH findings, fix if straightforward, otherwise document clearly.
153
+ > Read `.devlyn/fix-batch.round-<N>.json` and `pipeline.state.json`.
154
+ >
155
+ > **First, re-ground on the contract.** Open `source.spec_path` (or `source.criteria_path`) and read the sections/anchors referenced by each finding's `criterion_ref`. **Spec/criteria are higher authority than findings** — do not narrow or reinterpret required behavior to satisfy a finding. If a finding hint conflicts with explicit spec text (e.g., a glob/pattern like `**/SKILL.md`, a cardinality, a flag's documented behavior), preserve the spec semantics and fix only the implementation defect. Non-contradictory, backward-compatible enhancements that preserve required default behavior are allowed (e.g., respecting `NO_COLOR` while still defaulting to colored when unset). If a finding **truly contradicts** the spec, halt that finding's fix, log the conflict in `.devlyn/fix-batch.round-<N>.log.md`, and leave the finding `open` — the conflict surfaces in the final report rather than silently narrowing the contract.
156
+ >
157
+ > **Then fix every listed finding at the root cause.** If multiple findings touch the same symbol, produce **one consolidated change**. Prefer editing/replacing existing code over adding new machinery; **do not leave parallel near-duplicate helpers/functions**. When return-shape pressure appears (one finding needs a richer return value than another), broaden the existing helper's return object — don't create a second variant.
158
+ >
159
+ > Read each referenced `file:line`, implement the fix, run tests. No workarounds (`any`, `@ts-ignore`, silent catches, hardcoded values). Raw failure detail: `.devlyn/build_gate.log.md` / `.devlyn/browser_validate.log.md`. When a previously-failed criterion is now satisfied, clear `failed_by_finding_ids`, set `status: "implemented"`, append an `evidence` record.
496
160
 
497
161
  **After the agent completes**:
498
- 1. If CRITICAL issues were found and fixed, this is expected continue
499
- 2. If CRITICAL issues remain unfixed, log a warning in the final report
500
- 3. **Checkpoint**: Run `git add -A && git commit -m "chore(pipeline): security review complete"` if there are changes
162
+ 1. Checkpoint: `git add -A && git commit -m "chore(pipeline): fix round <N> (<triggered_by>)"`.
163
+ 2. Increment `state.rounds.global`.
164
+ 3. Route back: `build_gate` PHASE 1.4; `browser_validate` PHASE 1.5; **`evaluate` / `critic` → PHASE 2 (re-EVAL)**. All post-EVAL findings flow back through EVAL.
165
+ 4. **After re-EVAL returns PASS/PASS_WITH_ISSUES with `triggered_by == "critic"`**: re-run PHASE 3 CRITIC once before proceeding to DOCS. This verifies the fix didn't introduce new design/security issues the first CRITIC would have caught. Subsequent fix-loop rounds triggered from this re-CRITIC follow the same rule (bounded by `state.rounds.max_rounds`).
501
166
 
502
- ## PHASE 6: CLEAN (skippable)
167
+ ## PHASE 3: CRITIC (findings-only, route-gated)
503
168
 
504
- Skip if `--skip-clean` was set.
169
+ Skip if `state.route.selected == "fast"` OR `critic` in `state.route.bypasses`.
505
170
 
506
- Spawn a subagent using the Agent tool with `mode: "bypassPermissions"`.
171
+ One post-EVAL critic pass with two sub-concerns:
172
+ - **Design sub-pass** — "would a staff engineer block this PR?" (cold read, any finding → `NEEDS_WORK`). Always Claude.
173
+ - **Security sub-pass** — OWASP-style audit with mandatory dependency audit when any dep manifest OR lockfile changed (`package.json`, `requirements.txt`, `package-lock.json`, `pnpm-lock.yaml`, `yarn.lock`, `Pipfile.lock`, `poetry.lock`, `Cargo.toml`, `Cargo.lock`, `go.mod`, `go.sum`). On `--engine auto`: **Dual** (Claude + Codex parallel, merged). On others: single model per route.
507
174
 
508
- Agent prompt pass this to the Agent tool:
175
+ Hygiene concerns (unused imports, dead code) live in EVAL's `hygiene.*` findings at LOW severity, not a separate sub-pass here.
509
176
 
510
- Scan the codebase for dead code, unused dependencies, and code hygiene issues in recently changed files. Focus on: unused imports, unreachable code paths, unused variables, dependencies in package.json that are no longer imported. Keep the scope tight — only clean what's related to recent work. Remove what's confirmed dead, leave anything ambiguous.
511
-
512
- **After the agent completes**:
513
- 1. **Checkpoint**: Run `git add -A && git commit -m "chore(pipeline): cleanup complete"` if there are changes
177
+ **Before spawn**: capture `phase_pre_sha = git rev-parse HEAD` `state.phases.critic.pre_sha`.
514
178
 
515
- ## PHASE 7: DOCS (skippable)
179
+ **Spawn**: per `<engine_routing_convention>`. Prompt body: **`references/phases/phase-3-critic.md`**.
516
180
 
517
- Skip if `--skip-docs` was set.
518
-
519
- Spawn a subagent using the Agent tool with `mode: "bypassPermissions"`.
520
-
521
- Agent prompt — pass this to the Agent tool (include the original task description from `<pipeline_config>` so the agent can detect spec paths):
181
+ **After the agent completes**:
182
+ 1. Enforce `<post_eval_invariant>`: `git diff --name-only <phase_pre_sha> -- ':!.devlyn/**'` — non-empty → revert + emit invariant finding + route to fix loop.
183
+ 2. Read `state.phases.critic.verdict` (WORSE of design/security sub-verdicts):
184
+ - `PASS` → PHASE 4.
185
+ - `PASS_WITH_ISSUES` (security LOW only; design must be zero) terminal; PHASE 4.
186
+ - `NEEDS_WORK` / `BLOCKED` → PHASE 2.5 with `triggered_by: "critic"`.
522
187
 
523
- You are the Docs phase of the auto-resolve pipeline. You have two jobs, in this order.
188
+ ## PHASE 4: DOCS (doc-file mutations only)
524
189
 
525
- **Job 1 Roadmap Sync** (run first, only if this task implemented a roadmap item)
190
+ Skip if `docs` in `state.route.bypasses` OR `state.route.selected == "fast"`.
526
191
 
527
- The ideate skill produces specs at `docs/roadmap/phase-N/{id}-{slug}.md` and tracks them in `docs/ROADMAP.md`. When auto-resolve finishes a task for one of those specs, the index lies until someone flips it — and nobody does, so it rots. Your job is to flip it.
192
+ Spawn Claude `Agent` (`mode: "bypassPermissions"`). Include original task description. Prompt: "Two jobs:
528
193
 
529
- 1. **Detect whether this task was a spec implementation.** Look at the original task description you were passed. Match against this regex: `docs/roadmap/phase-\d+/[^\s"'\`)]+\.md`. If there is no match, or if `docs/ROADMAP.md` does not exist in the repo, Job 1 is a no-op — skip straight to Job 2.
530
- 2. **Sanity-check against the diff.** Run `git diff main --stat` (or `git diff HEAD~N --stat` if on main). If the diff is empty or contains only doc changes, the build phase produced nothing — do NOT flip any status. Leave Job 1 untouched and continue to Job 2.
531
- 3. **Read the spec file** at the matched path. If its frontmatter already has `status: done`, Job 1 is already done skip to Job 2. Otherwise:
532
- - Set `status: done` in the frontmatter.
533
- - Add a `completed: YYYY-MM-DD` field (use today's date from `date +%Y-%m-%d`).
534
- - Do not change any other fields, and do not touch the body of the spec.
535
- 4. **Update `docs/ROADMAP.md`.** Find the row whose `#` column matches the spec's `id` (e.g., row starting `| 2.3 |`). Change its Status column to `Done`. Do not touch any other row, and do not reformat the table.
536
- 5. **Check whether the phase is now fully Done.** Read every row of the phase's table (the one containing the just-flipped row). If every row's Status is `Done`, archive the phase:
537
- - Cut the phase's `## Phase N: …` heading and table out of the active section of ROADMAP.md.
538
- - If no `## Completed` section exists at the bottom of the file, create one just above end-of-file (below Decisions if Decisions exists).
539
- - Add a `<details>` block for the phase inside Completed, using the format defined in the devlyn:ideate skill's Context Archiving section. Pull each item's completion date from its spec file's `completed:` frontmatter; if a spec has none, use today's date.
540
- - Item spec files stay on disk — do not delete them. Only the index row moves.
541
- 6. **Report.** In your summary, say explicitly what you did: "Flipped spec 2.3 to done, updated ROADMAP.md row." And if applicable: "Phase 2 was fully Done — archived to Completed block."
194
+ **Job 1 Roadmap sync**: if task matched `docs/roadmap/phase-\d+/[^\s\"']+\.md` and `git diff <state.base_ref.sha> --stat` touches non-doc files:
195
+ 1. Read the spec. If `status: done` already, skip to Job 2.
196
+ 2. Set `status: done` + `completed: <today>` in frontmatter. Do not touch body.
197
+ 3. Update `docs/ROADMAP.md`: find row matching spec id; change Status to `Done`.
198
+ 4. If phase now fully Done: archive to `## Completed <details>` block at bottom (format per `devlyn:ideate#context-archiving`). Item spec files stay on disk.
542
199
 
543
- **Safety invariants**violating any of these means stop Job 1 and report it:
544
- - Never flip a spec to `done` without a non-empty `git diff` touching non-doc files.
545
- - Never flip multiple specs in one run — one task, one spec.
546
- - Never edit a row whose `#` doesn't exactly match the spec's `id`.
547
- - Never delete spec files.
200
+ **Job 2General doc sync**: update docs referencing changed APIs/features/behaviors. Use `git log --oneline -20` + `git diff <state.base_ref.sha>`. Preserve forward-looking content.
548
201
 
549
- **Job 2 General doc sync**
202
+ **Safety**: never flip a spec `done` without a non-empty non-doc diff; never flip multiple specs in one run; never touch files outside the doc-file allowlist."
550
203
 
551
- Synchronize the rest of the documentation with recent code changes. Use `git log --oneline -20` and `git diff main` to understand what changed. Update any docs that reference changed APIs, features, or behaviors. Do not create new documentation files unless the changes introduced entirely new features with no existing docs. Preserve all forward-looking content: future plans, visions, open questions. (Job 1 already handled the roadmap index — don't second-guess it here.)
204
+ **Before spawn**: capture `phase_pre_sha = git rev-parse HEAD` `state.phases.docs.pre_sha`.
552
205
 
553
206
  **After the agent completes**:
554
- 1. **Checkpoint**: Run `git add -A && git commit -m "chore(pipeline): docs updated"` if there are changes
555
-
556
- ## PHASE 8: FINAL REPORT
207
+ 1. Enforce allowlist: `git diff --name-only <phase_pre_sha> -- ':!.devlyn/**'` — any non-allowlisted path revert + emit `invariant.post-eval-code-mutation` + route to PHASE 2.5 with `triggered_by: "docs"`.
208
+ 2. If allowlist honored and diff non-empty: `git add -A && git commit -m "chore(pipeline): docs updated"`.
557
209
 
558
- After all phases complete:
210
+ ## PHASE 5: FINAL REPORT + ARCHIVE
559
211
 
560
- 1. Clean up temporary files:
561
- - Delete the `.devlyn/` directory entirely (contains done-criteria.md, BUILD-GATE.md, EVAL-FINDINGS.md, BROWSER-RESULTS.md, CHALLENGE-FINDINGS.md, screenshots/, playwright temp files)
562
- - Kill any dev server process still running from browser validation
563
-
564
- 2. Run `git log --oneline -10` to show commits made during the pipeline
565
-
566
- 3. Present the report:
212
+ 1. **Terminal verdict**: run `python3 scripts/terminal_verdict.py` (implements the precedence in `references/pipeline-routing.md#terminal-state-algorithm`; prints verdict, exits 0/1/2/3 for PASS/PASS_WITH_ISSUES/NEEDS_WORK/BLOCKED).
567
213
 
214
+ 2. **Render report**:
568
215
  ```
569
- ### Auto-Resolve Pipeline Complete
570
-
571
- **Task**: [original task description]
572
- **Engine**: [auto / codex / claude — if auto, note which phases used which model]
573
-
574
- **Pipeline Summary**:
575
- | Phase | Status | Notes |
576
- |-------|--------|-------|
577
- | Build (team-resolve) | [completed] | [brief summary; engine that ran it] |
578
- | Build gate | [completed / skipped / FAIL after N rounds] | [project types detected, commands run, pass/fail per command] |
579
- | Browser validate | [completed / skipped / auto-skipped] | [verdict, tier used, console errors, flow results] |
580
- | Evaluate | [PASS/NEEDS WORK after N rounds] | [verdict + key findings] |
581
- | Fix rounds | [N rounds / skipped] | [what was fixed] |
582
- | Simplify | [completed / skipped] | [changes made] |
583
- | Review (team) | [completed / skipped] | [findings summary; per-role engines if --engine auto] |
584
- | Challenge | [PASS / NEEDS WORK] | [findings count, fixes applied] |
585
- | Security review | [completed / skipped / auto-skipped] | [findings or "no security-sensitive changes"] |
586
- | Clean | [completed / skipped] | [items cleaned] |
587
- | Docs (update-docs) | [completed / skipped] | [docs updated] |
588
-
589
- **Evaluation Rounds**: [N] of [max-rounds] used (shared budget across PHASE 1.4-fix and PHASE 2.5)
590
- **Final Verdict**: [last evaluation verdict, or "BUILD GATE FAILED — code does not compile" if PHASE 1.4 exhausted the round budget before PHASE 2 ran]
591
-
592
- **Commits created**:
593
- [git log output]
594
-
595
- **What to do next**:
596
- - Review the changes: `git diff main`
597
- - If satisfied, squash pipeline commits: `git rebase -i main` (combine the chore commits into meaningful ones)
598
- - If not satisfied, run specific fixes: `/devlyn:team-resolve [specific issue]`
599
- - For a final human review: `/devlyn:team-review`
216
+ ### Auto-Resolve Complete — run <run_id>
217
+
218
+ Task: <original task>
219
+ Engine: <engine> (downgraded: <reason or no>)
220
+ Route: <selected> (user_override: <t/f>)
221
+ Stage A: <reasons>
222
+ Stage B LITE: <no escalation | escalated from X — reason>
223
+
224
+ Terminal verdict: <PASS / PASS_WITH_ISSUES / NEEDS_WORK / BLOCKED>
225
+ <banner if applicable: "⚠ BUILD GATE EXHAUSTED" / "⚠ EVAL EXHAUSTED open findings: <list file:line>" />
226
+
227
+ Pipeline summary:
228
+ | Phase | Verdict | Notes |
229
+ |-------|---------|-------|
230
+ | BUILD | <v> | <engine, solo/team> |
231
+ | BUILD GATE | <v> | <project types, commands> |
232
+ | BROWSER | <v / skipped no web> | <tier, flow> |
233
+ | EVAL (round <N>) | <v> | <finding count by severity> |
234
+ | FIX ROUNDS | <N of max> | <triggered_by history> |
235
+ | CRITIC | <v / skipped-route / skipped-bypass> | <design: N, security: N, dep-audit: ran/skipped> |
236
+ | DOCS | <completed / skipped> | <specs flipped, roadmap archived> |
237
+
238
+ Guardrails bypassed: <state.route.bypasses or "none">
239
+
240
+ Commits: <git log --oneline from state.base_ref.sha>
241
+
242
+ Audit trail: .devlyn/runs/<run_id>/
243
+
244
+ Next steps:
245
+ - Review: git diff <base_ref.sha>
246
+ - Squash: git rebase -i <base_ref.sha>
247
+ - Re-run fixes: /devlyn:auto-resolve "<narrower task>"
600
248
  ```
601
249
 
602
- </pipeline_workflow>
250
+ 3. **Archive**: run `python3 scripts/archive_run.py` (implements `references/pipeline-state.md#archive-contract`; moves per-run artifacts into `.devlyn/runs/<run_id>/`, best-effort prunes to last 10 completed runs).
251
+
252
+ 4. Kill dev server from PHASE 1.5 if still running.