@dtlabs/ai-toolkit 0.5.0 → 0.7.0

package/.claude/agents/define-feature.md

@@ -6,7 +6,75 @@ model: sonnet
 
 # Define Feature
 
-Helps the user define a new feature by asking relentless, focused questions — then synthesizes the answers into a `feature.md` file ready for the `/implement-feature` pipeline.
+Helps the user define a new feature by first **reading everything the repository already knows**, then asking only the questions whose answers are genuinely missing — and finally synthesizing the result into a `feature.md` file ready for the `/implement-feature` pipeline.
+
+**Core principle: never ask what the repo can already answer.** Every question you ask must be one you could not resolve by reading `AGENTS.md`, an existing backlog/ticket, the existing feature docs, or the code itself. Redundant questions are a defect.
+
+---
+
+## Input
+
+The user MAY provide a **source reference** as an argument — treat it as authoritative input, not as a starting point to re-derive from scratch:
+
+- A path to a backlog entry or section (e.g. `docs/token-optimization-backlog.md → OPT-09`)
+- A path to a ticket, spec, or design note
+- A free-text description of the feature
+- Nothing at all (fully interactive definition)
+
+If a source reference is given, your job is to **ingest it, pre-fill the feature draft, and grill only on the real gaps** — not to ask the user to restate what the source already says.
+
+---
+
+## Phase 0 — Context Ingestion (MANDATORY — before any question)
+
+Do NOT ask the user anything until you have read the available context. Silently gather, then build a pre-filled draft.
+
+### 0a. Read project conventions
+
+- Read `AGENTS.md` from the project root if it exists (tech stack, directory structure, patterns, constraints). This answers most stack/structure/permission questions without asking.
+- If `AGENTS.md` is absent, note it — you will infer conventions from the code and may confirm the stack in one consolidated question.
+
+### 0b. Ingest the source reference (if provided)
+
+- Read the referenced file/section fully. Extract every field it already provides: objective, scope, out-of-scope, affected files, acceptance criteria, dependencies, effort/risk.
+- Follow obvious pointers (e.g. a backlog entry that says "deriva da `docs/token-optimization.md`" → read that too).
+
+### 0c. Study existing features and code
+
+- List `docs/features/` and `internal_docs/features/` (whichever exists) to learn the house style of prior `feature.md` files and to detect overlap/dependencies with existing work.
+- Read the code the feature will touch (files named in the source, or found via Grep/Glob) enough to understand the current behavior being changed.
+
+### 0d. Classify the feature type — this drives which questions are asked
+
+Decide the primary type from the evidence gathered:
+
+| Type | Signals | Rounds that DON'T apply |
+|------|---------|-------------------------|
+| **business/UI** | end-user pages, forms, actors, CRUD on domain entities | (all rounds apply) |
+| **internal/technical** | refactor, tooling, script, workflow, agent, build/infra, optimization | Actors, Core-flow-as-page, Data model, Roles/permissions |
+| **library/API** | public functions/endpoints, no UI, consumed by other code | Core-flow-as-page, Roles/permissions (unless auth is the point) |
+
+Record the type. In Phase 2 you will **skip inapplicable rounds entirely** and mark their template sections `N/A — <type> feature`.
+
+### 0e. Build and present the pre-filled draft
+
+Assemble a draft of the feature fields from 0a–0c and show it to the user **before grilling**:
+
+```
+📋 Pre-filled from: <source reference or "no source — interactive">
+   Feature type: <business/UI | internal/technical | library/API>
+──────────────────────────────────────────────────
+Objective       : <from source, or "❓ unknown">
+Scope           : <from source, or "❓ unknown">
+Out of scope    : <from source, or "❓ unknown">
+Files affected  : <from source/code, or "❓ unknown">
+Acceptance      : <from source, or "❓ unknown">
+Dependencies    : <from source, or "❓ unknown">
+──────────────────────────────────────────────────
+Gaps I still need to resolve: <list only the ❓ items + genuine ambiguities>
+```
+
+Ask the user to **confirm or correct the draft**. Then grill ONLY on the `❓` gaps and ambiguities — skip every field already answered by the source.
 
 ---
 
@@ -30,11 +98,16 @@ Example: "Supplier Onboarding" → `FTR-003-supplier-onboarding`
 
 ---
 
-## Phase 2 — Grilling
+## Phase 2 — Grilling (only the gaps)
+
+**Objective**: resolve the `❓` gaps and ambiguities identified in Phase 0 — nothing already answered.
 
-**Objective**: surface every assumption, gap, and edge case before writing a single line of spec.
+**Hard rules before asking any question:**
+1. **Never ask what Phase 0 already answered.** If the source, `AGENTS.md`, existing features, or the code already provides a field, do NOT ask about it — at most ask a single confirmation if genuinely ambiguous.
+2. **Skip rounds that don't apply to the feature type** (from Phase 0d). For an internal/technical feature, skip Actors, page-based Core Flow, Data model, and Roles/permissions entirely and mark those template sections `N/A — technical feature`.
+3. **One dimension at a time** — use `AskUserQuestion` per round; adapt follow-ups to prior answers.
 
-Ask questions in rounds. Each round focuses on a dimension. Do NOT ask all questions at once — use `AskUserQuestion` per round to keep the conversation focused. Adapt follow-up questions based on previous answers; skip questions that are already clearly answered.
+Below are the possible rounds. Run a round **only if** it applies to the feature type AND it contains at least one unresolved gap. If a round has no open gap, skip it silently.
 
 ### Round 1 — The problem
 
@@ -49,15 +122,21 @@ Follow-up if vague:
 
 ### Round 2 — The core flow
 
-> "Walk me through the main scenario step by step — from the moment the user opens the page to the moment they're done."
+**Applies to:** all types — but phrase it to the type.
+- business/UI: *"Walk me through the main scenario step by step — from the moment the user opens the page to the moment they're done."*
+- internal/technical or library/API: *"Walk me through the main execution path — what triggers it, what it does step by step, and what the end state is."* (There is no page; do not ask page questions.)
 
-Goal: extract the happy path in concrete user actions.
+Goal: extract the happy path in concrete steps.
 
-Follow-up:
+Follow-up (business/UI):
 - "What does the user see first?"
 - "What data do they need to input or select?"
 - "What happens after they submit/save?"
-- "How does the system respond?"
+
+Follow-up (technical/API):
+- "What is the entry point (function, script, command, event)?"
+- "What are the inputs and where do they come from?"
+- "What is the observable output or side effect when it completes?"
 
 ### Round 3 — Boundaries
 
@@ -81,6 +160,8 @@ Follow-up:
 
 ### Round 5 — Data and entities
 
+**Applies to:** business/UI and library/API features that manage domain data. **Skip for internal/technical/refactor/tooling features that create no domain entities** — mark the Data Model section `N/A — technical feature`.
+
 > "What data does this feature create, read, update, or delete?"
 
 Follow-up:
@@ -90,6 +171,8 @@ Follow-up:
 
 ### Round 6 — Roles and permissions
 
+**Applies to:** business/UI features (and API features where authorization is in scope). **Skip for internal/technical features with no end-user roles** — mark the Roles and Permissions section `N/A — technical feature`.
+
 > "Who can do what? Are there different permission levels within this feature?"
 
 Follow-up:
@@ -226,9 +309,10 @@ For each entity: name, key fields (name, type, constraints), relationships.
 ### Writing rules
 
 - Write in **English** (regardless of the language used during grilling)
-- Derive content **strictly from the user's answers** — do not invent requirements
+- Derive content from **the source reference (Phase 0) plus the user's answers** — do not invent requirements
+- For sections that don't apply to the feature type, write `N/A — <type> feature` instead of leaving them blank or inventing content (e.g. Actors / Data Model / Roles for an internal/technical feature)
 - Mark open questions explicitly rather than guessing
-- Acceptance criteria must be **testable** (Given/When/Then)
+- Acceptance criteria must be **testable** (Given/When/Then) — for technical features, express them as observable/verifiable checks (e.g. "grep returns nothing", "build passes", "output file exists")
 - Be concrete and specific — avoid vague statements like "the system should be fast"
 
 ---
@@ -258,9 +342,12 @@ Next step: run /implement-feature docs/features/{PREFIX}-{slug}/feature.md
 
 ## Guidelines
 
-- **Grill relentlessly** — vague answers get follow-up questions, not acceptance
-- **One dimension at a time** — do not overwhelm the user with all questions at once
-- **Challenge contradictions** — if answers conflict, surface it before writing
-- **Do not invent** — if something is unclear and the user can't answer, record it as an open question
-- **Create the folder** if it doesn't exist — do not ask the user to do it manually
-- **Always write in English** — feature.md is input to English-language downstream agents
+- **Read before you ask** — Phase 0 is mandatory; never ask what `AGENTS.md`, the source reference, existing feature docs, or the code already answers. A redundant question is a defect.
+- **Match questions to the feature type** — skip Actors / Data model / Roles / page-flow rounds for internal/technical features; do not force business-feature questions onto a refactor or tooling task.
+- **Confirm the draft, then grill the gaps** — present the pre-filled draft first; only unresolved `❓` items and genuine ambiguities become questions.
+- **Grill relentlessly on real gaps** — vague answers to open questions get follow-ups, not acceptance.
+- **One dimension at a time** — do not overwhelm the user with all questions at once.
+- **Challenge contradictions** — if answers conflict with each other or with the source, surface it before writing.
+- **Do not invent** — if something is unclear and the user can't answer, record it as an open question.
+- **Create the folder** if it doesn't exist — do not ask the user to do it manually.
+- **Always write in English** — feature.md is input to English-language downstream agents.

package/.claude/agents/developer-testing.md

@@ -78,7 +78,7 @@ Follow all conventions EXACTLY.
 4. **Create test project/file** if it doesn't exist (following project conventions)
 5. **Write test helpers** (custom factories, builders, mocks) if needed
 6. **Implement tests** following AAA pattern
-7. **Verify** by running the test command from AGENTS.md
+7. **Syntax check only (optional)** — you MAY run a build to confirm the test files compile (e.g. `dotnet build`, `npx tsc --noEmit`). **Do NOT run the test suite** (`dotnet test`, `npm test`, etc.) — the orchestrator runs the full suite once, after all agents finish. Running it here duplicates a multi-minute build per agent and contends for build artifacts.
 
 ---
 
@@ -102,13 +102,13 @@ When all assigned tasks are complete, return a **structured summary** to the orc
 Tasks completed: [list]
 Files created:   [list with path only]
 Files modified:  [list with path only]
-Test result:     ✅ N/N passed / ❌ N failed — [failing test names if any]
+Build check:     ✅ compiles / ❌ compile error — [first error line if any] / ⏭️ not run
 Notes:           [blocking issues, skipped cases, TODO items]
 ─────────────────────────────────────────────
 ```
 
 - **Never dump test code** into the summary
-- If tests FAILED, include only the failing test names and first error line
+- Do NOT report test-run results — you do not run the suite; the orchestrator runs it once after all agents finish. Report only whether the test files compile.
 
 ---
 
@@ -119,4 +119,4 @@ Notes:           [blocking issues, skipped cases, TODO items]
 - **No testing of trivial code** — getters/setters, data classes, config files
 - **Mock external dependencies** — databases, HTTP clients, file systems
 - **Use the project's preferred mocking approach** — as defined in AGENTS.md
-- **After implementation**: run the test command to verify all tests pass
+- **Do NOT run the test suite** — the orchestrator runs the full suite once, after every agent has finished writing its files. A build for a quick syntax check is fine; running the tests here wastes minutes on a duplicate build and can lock build artifacts.

package/.claude/agents/validate-feature-docs.md

@@ -121,9 +121,11 @@ For each document with gaps, invoke the corresponding agent with a **targeted re
 
 ---
 
-## Phase 6 — Write Validation Report
+## Phase 6 — Write Validation Report (MANDATORY — your final action)
 
-**Always execute this phase**, regardless of whether gaps were found. The report is required even when validation is clean.
+**This is not optional and not the same as the Phase 4 gap report.** The coverage report you produced in Phase 4 is on-screen text; it is NOT the deliverable. Your job is not complete until the file `{PREFIX}-Validation-Report.md` exists **on disk**. You MUST call the `Write` tool to create it — even when validation is clean and zero gaps were found. Returning a summary without having called `Write` is a failure.
+
+Execute this phase **last, always**, regardless of outcome. Downstream agents read this file as a hard precondition; if it is missing, the entire pipeline aborts.
 
 Write `{PREFIX}-Validation-Report.md` in the same directory:
 
@@ -166,6 +168,7 @@ Do NOT guess how to fill ambiguous gaps. Ask first, then revise only with a clea
 
 ## Guidelines
 
+- **Always call `Write` for the Validation Report before finishing** (Phase 6) — the on-screen gap report is not the deliverable; the file on disk is. This is your last action, clean or not.
 - **Do not regenerate documents from scratch** — targeted revisions only
 - **Be specific about gaps** — point to the exact section, claim, and what is missing
 - **Stop after 3 revision cycles** — if coverage is still incomplete, report remaining gaps and exit

package/.claude/skills/define-feature/SKILL.md

@@ -1,13 +1,15 @@
 ---
-description: "Define Feature — grills the user to define a new feature, then writes feature.md in docs/features/FTR-XXX-slug/. Usage: /define-feature"
+description: "Define Feature — reads existing repo context, then grills only on the real gaps and writes feature.md in docs/features/FTR-XXX-slug/. Usage: /define-feature [source reference]"
 disable-model-invocation: true
 ---
 
-Spawn the `define-feature` agent with no additional arguments:
+Spawn the `define-feature` agent, passing through whatever the user provided as a source reference (a backlog entry, ticket path, spec path, or free-text description). If the user gave nothing, pass an empty source.
 
 ```
 subagent_type: define-feature
-prompt: Start the feature definition session.
+prompt: |
+  Start the feature definition session.
+  Source reference (may be empty): {{user arguments verbatim}}
 ```
 
-Do not add anything else. The agent handles discovery, grilling, and file creation autonomously.
+The agent runs Phase 0 (context ingestion) first — it reads AGENTS.md, the source reference, existing feature docs, and the relevant code, then pre-fills a draft and grills ONLY on the genuine gaps. Do not add instructions telling it to skip discovery.

package/.claude/workflows/pm-phase1.js

@@ -103,6 +103,7 @@ if (discoveryResult.needs_tech_spec) {
 phase('Validation')
 
 let validationSummary = 'skipped'
+let lastValText       = ''
 const MAX_CYCLES = 3
 
 for (let cycle = 1; cycle <= MAX_CYCLES; cycle++) {
@@ -115,6 +116,7 @@ for (let cycle = 1; cycle <= MAX_CYCLES; cycle++) {
   // valResult is the agent's text output — check for gap indicators
   // Use only 'MISSING:' as the gap signal: 'gaps found' also matches 'ZERO GAPS FOUND'
   const resultText = typeof valResult === 'string' ? valResult : JSON.stringify(valResult)
+  lastValText      = resultText
   const hasGaps    = resultText.includes('MISSING:')
 
   if (!hasGaps) {
@@ -139,6 +141,56 @@ for (let cycle = 1; cycle <= MAX_CYCLES; cycle++) {
   }
 }
 
+// ── Guarantee the Validation Report exists on disk ───────────────────────────
+// The validate-feature-docs agent (haiku) is instructed to write the report in its
+// Phase 6, but this is LLM-dependent and unreliable when validation is clean. Downstream
+// pm-phase2 (generate-work-breakdown) treats the report as a hard precondition, so its
+// absence aborts the whole pipeline. This step deterministically ensures the file exists.
+if (validationSummary !== 'skipped') {
+  const validationReportPath = `${feature_dir}/${prefix}-Validation-Report.md`
+  await agent(
+    `Ensure the Validation Report file exists on disk for this feature delivery run.
+
+File path: ${validationReportPath}
+
+Steps:
+1. Check whether the file already exists (use Read or Glob).
+2. If it ALREADY EXISTS: do nothing, return "exists".
+3. If it DOES NOT EXIST: write it now using the Write tool, based on the validation outcome below.
+   Get the current date via Bash: run \`date -u +"%Y-%m-%d"\`.
+
+Validation outcome summary: ${validationSummary}
+
+Validation agent output (source of truth for gaps found/resolved):
+──────────────────────────────────────────────────
+${lastValText.slice(0, 4000)}
+──────────────────────────────────────────────────
+
+File format to write (fill from the outcome above; if the run was clean, mark both documents Clean with 0 gaps):
+
+# Validation Report — ${prefix}
+
+## Summary
+| Document | Gaps found | Gaps resolved | Status |
+|----------|-----------|--------------|--------|
+| ${prefix}-Requirements.md | N | N | Clean / Gaps remain |
+| ${prefix}-Tech-Spec.md    | N | N | Clean / Gaps remain |
+
+## Gaps found and resolved
+(derive from the validation output above; write "(none)" if clean)
+
+## Remaining gaps (if any)
+(list any unresolved gaps, or "(none)")
+
+## Validation date
+{date from Bash}
+
+Return "written" if you created the file, or "exists" if it was already present.`,
+    { label: 'ensure-validation-report', phase: 'Validation' }
+  )
+  log(`Validation report ensured at ${validationReportPath}`)
+}
+
 // ── Write process-log (phase 1 snapshot) ─────────────────────────────────────
 const reqEntry   = tokenLedger.find(e => e.agent === 'generate-requirements')
 const specEntry  = tokenLedger.find(e => e.agent === 'generate-tech-spec')

package/.claude/workflows/pm-phase3.js

@@ -1,10 +1,11 @@
 export const meta = {
   name: 'pm-phase3',
-  description: 'Feature delivery phase 3: parse WB → dispatch specialist agents per US (commit per phase) → single architect review → PR → actuals.',
+  description: 'Feature delivery phase 3: parse WB → dispatch specialist agents per US (developer-backend/frontend/testing + review-solution per US, max 2 rework cycles, git commit per US) → remediation → PR → Token-Estimate actuals → Effort-Estimate actuals.',
   phases: [
-    { title: 'Parse',          detail: 'Parse Work Breakdown CSV into structured phases' },
-    { title: 'Implementation', detail: 'Dispatch developer agents per phase, commit each one' },
-    { title: 'Review',         detail: 'Single architect review on full feature diff' },
+    { title: 'Parse',          detail: 'Parse Work Breakdown into structured implementation phases' },
+    { title: 'Implementation', detail: 'Dispatch developer agents per US, review, commit' },
+    { title: 'Test',           detail: 'Single test run after all phases complete' },
+    { title: 'Remediation',    detail: 'Fix OPEN issues from Issues Register' },
     { title: 'PR',             detail: 'Push branch, create PR, update registry' },
     { title: 'Actuals',        detail: 'Write Token-Estimate and Effort-Estimate actuals' },
   ],
@@ -17,10 +18,12 @@ const branchMatch = argStr.match(/--branch\s+(\S+)/)
 const branch      = branchMatch ? branchMatch[1] : null
 
 const tokenLedger = []
+const escalations = []
 
 // ── Parse Work Breakdown CSV (deterministic, no AI) ───────────────────────────
 phase('Parse')
 
+// Derive feature_dir and prefix from featurePath (e.g. "docs/features/FTR-004-.../feature.md")
 const featureDir  = featurePath.replace(/\/[^/]+$/, '')
 const prefixMatch = featureDir.match(/([A-Z]+-\d+)/)
 const prefix      = prefixMatch ? prefixMatch[1] : 'FTR-000'
@@ -37,7 +40,7 @@ const csvContent = await agent(
 const rows = csvContent
   .split('\n')
   .map(l => l.trim())
-  .filter(l => l && !l.startsWith('phase_id'))
+  .filter(l => l && !l.startsWith('phase_id'))  // skip header and empty lines
   .map(l => {
     const [phase_id, phase_title, commit_message, depends_on, task_id, task_title, domain, agent_type] = l.split('|')
     return { phase_id, phase_title, commit_message, depends_on: depends_on || '', task_id, task_title, domain, agent_type }
@@ -52,7 +55,7 @@ for (const row of rows) {
       phase_id:       row.phase_id,
       title:          row.phase_title,
       commit_message: row.commit_message,
-      depends_on:     row.depends_on.split(' ').filter(Boolean),
+      depends_on:     row.depends_on.split(' ').filter(Boolean),  // [] for empty
       impl_tasks:     [],
       test_tasks:     [],
     })
@@ -85,9 +88,11 @@ const allPhases = Array.from(phaseMap.values()).map(p => ({
 }))
 
 const wb = { prefix, feature_dir: featureDir, phases: allPhases }
+
 log(`CSV parsed: ${rows.length} tasks → ${wb.phases.length} phases for ${wb.prefix}`)
 
-// Build execution waves: phases whose dependencies are all satisfied run in parallel
+// Build execution waves: phases whose dependencies are all satisfied can run in parallel.
+// Wave 0 = no deps, Wave 1 = depends only on Wave 0 phases, etc.
 const buildWaves = (phases) => {
   const done   = new Set()
   const waves  = []
@@ -95,6 +100,7 @@ const buildWaves = (phases) => {
   while (remaining.length > 0) {
     const ready = remaining.filter(p => p.depends_on.every(d => done.has(d)))
     if (ready.length === 0) {
+      // Circular or unresolvable deps — run the rest sequentially to avoid deadlock
       waves.push(remaining)
       break
     }
@@ -108,71 +114,140 @@ const buildWaves = (phases) => {
 const waves = buildWaves(wb.phases)
 log(`Execution plan: ${waves.length} wave(s) — ${waves.map((w, i) => `wave${i+1}:[${w.map(p => p.phase_id).join(',')}]`).join(' ')}`)
 
-let phasesDone = 0
-const usPassed = []
+let phasesDone   = 0
+const usPassed   = []
+const usEscalated = []
+let issuesPath   = ''
+let issuesOpen   = 0
 
 // ── Implementation ────────────────────────────────────────────────────────────
 phase('Implementation')
 
-// executePhase: impl_groups (parallel) → test_groups (parallel) → commit
-// No inline review — review runs once after all phases complete.
+const REVIEW_SCHEMA = {
+  type: 'object',
+  properties: {
+    has_critical: { type: 'boolean' },
+    findings:     { type: 'string' },
+    issues_md:    { type: 'string' },
+  },
+  required: ['has_critical'],
+}
+
+// executePhase: runs impl_groups → test_groups → review → rework → commit for one phase
 const executePhase = async (implPhase) => {
   log(`Starting phase ${implPhase.phase_id}: ${implPhase.title}`)
 
-  // Step 1 — impl groups in parallel (INFRA / BE / FE)
-  if (implPhase.impl_groups.length > 0) {
-    const beforeImpl = budget.spent()
-    await parallel(implPhase.impl_groups.map(group => () =>
-      agent(
-        `${featurePath} ${group.task_ids.join(',')}`,
-        {
-          agentType: group.agent_type,
-          label:     `${group.agent_type}:${implPhase.phase_id}`,
-          phase:     'Implementation',
-        }
-      )
-    ))
-    const implTokens = budget.spent() - beforeImpl
+  let reworkCycle  = 0
+  let passed       = false
+  let lastFindings = ''
+
+  while (reworkCycle <= 2 && !passed) {
+    const reworkSuffix = reworkCycle > 0
+      ? ` -- REWORK CYCLE ${reworkCycle}: address these critical findings:\n${lastFindings}`
+      : ''
+
+    // Step 1 — impl groups in parallel (INFRA / BE / FE)
+    if (implPhase.impl_groups.length > 0) {
+      const beforeImpl = budget.spent()
+      await parallel(implPhase.impl_groups.map(group => () =>
+        agent(
+          `${featurePath} ${group.task_ids.join(',')}${reworkSuffix}`,
+          {
+            agentType: group.agent_type,
+            label:     `${group.agent_type}:${implPhase.phase_id}${reworkCycle > 0 ? ':rework' + reworkCycle : ''}`,
+            phase:     'Implementation',
+          }
+        )
+      ))
+      const implTokens = budget.spent() - beforeImpl
+      tokenLedger.push({
+        agent: `${implPhase.impl_groups.map(g => g.agent_type).join('+')}:${implPhase.phase_id}${reworkCycle > 0 ? ':rework' + reworkCycle : ''}`,
+        model: 'sonnet',
+        phase_delta_tokens: implTokens,
+      })
+      log(`Phase ${implPhase.phase_id} impl groups done — ${implTokens} tokens`)
+    }
+
+    // Step 2 — test groups in parallel (TEST) — after impl groups
+    if (implPhase.test_groups.length > 0) {
+      const beforeTest = budget.spent()
+      await parallel(implPhase.test_groups.map(group => () =>
+        agent(
+          `${featurePath} ${group.task_ids.join(',')}${reworkSuffix}`,
+          {
+            agentType: group.agent_type,
+            label:     `${group.agent_type}:${implPhase.phase_id}${reworkCycle > 0 ? ':rework' + reworkCycle : ''}`,
+            phase:     'Implementation',
+          }
+        )
+      ))
+      const testTokens = budget.spent() - beforeTest
+      tokenLedger.push({
+        agent: `developer-testing:${implPhase.phase_id}${reworkCycle > 0 ? ':rework' + reworkCycle : ''}`,
+        model: 'sonnet',
+        phase_delta_tokens: testTokens,
+      })
+      log(`Phase ${implPhase.phase_id} test groups done — ${testTokens} tokens`)
+    }
+
+    // Step 3 — review-solution
+    const beforeReview = budget.spent()
+    const review = await agent(
+      `${featurePath} --scope ${implPhase.phase_id}`,
+      {
+        agentType: 'review-solution',
+        label:     `review-solution:${implPhase.phase_id}`,
+        phase:     'Implementation',
+        schema:    REVIEW_SCHEMA,
+      }
+    )
+    const reviewTokens = budget.spent() - beforeReview
     tokenLedger.push({
-      agent:              `${implPhase.impl_groups.map(g => g.agent_type).join('+')}:${implPhase.phase_id}`,
-      model:              'sonnet',
-      phase_delta_tokens: implTokens,
+      agent: `review-solution:${implPhase.phase_id}`,
+      model: 'sonnet',
+      phase_delta_tokens: reviewTokens,
     })
-    log(`Phase ${implPhase.phase_id} impl done — ${implTokens} tokens`)
+
+    if (review && !review.has_critical) {
+      passed = true
+      if (review.issues_md) {
+        await agent(
+          `Append the following WARNING/INFO findings to ${wb.feature_dir}/${prefix}-Issues.md (create the file if it does not exist, with a header "# Issues Register — ${prefix}"):\n\n${review.issues_md}`,
+          { label: `log-issues:${implPhase.phase_id}`, phase: 'Implementation' }
+        )
+      }
+    } else {
+      lastFindings = review?.findings || 'unspecified critical issues'
+      reworkCycle++
+      log(`Phase ${implPhase.phase_id}: critical findings — rework cycle ${reworkCycle}`)
+    }
   }
 
-  // Step 2 — test groups in parallel (TEST)
-  if (implPhase.test_groups.length > 0) {
-    const beforeTest = budget.spent()
-    await parallel(implPhase.test_groups.map(group => () =>
-      agent(
-        `${featurePath} ${group.task_ids.join(',')}`,
-        {
-          agentType: group.agent_type,
-          label:     `developer-testing:${implPhase.phase_id}`,
-          phase:     'Implementation',
-        }
-      )
-    ))
-    const testTokens = budget.spent() - beforeTest
-    tokenLedger.push({
-      agent:              `developer-testing:${implPhase.phase_id}`,
-      model:              'sonnet',
-      phase_delta_tokens: testTokens,
-    })
-    log(`Phase ${implPhase.phase_id} tests done — ${testTokens} tokens`)
+  if (!passed) {
+    log(`Phase ${implPhase.phase_id} ESCALATED after 2 rework cycles`)
+    escalations.push({ phaseId: implPhase.phase_id, reason: 'CRITICAL review findings unresolved after 2 rework cycles' })
+    usEscalated.push(implPhase.phase_id)
+  } else {
+    const commitMsg = implPhase.commit_message || `feat(${prefix}): implement ${implPhase.phase_id}`
+    await agent(
+      `Run these git commands in the repository root:\ngit add -A\ngit commit -m "${commitMsg}"\nIf there is nothing to commit, that is fine — just report success.`,
+      { label: `commit:${implPhase.phase_id}`, phase: 'Implementation' }
+    )
+    if (implPhase.phase_id !== 'INFRA') usPassed.push(implPhase.phase_id)
+    log(`Phase ${implPhase.phase_id} passed review and committed`)
   }
 
-  // Step 3 — commit this phase
-  const commitMsg = implPhase.commit_message || `feat(${prefix}): implement ${implPhase.phase_id}`
+  phasesDone++
+
+  // Persist the cumulative token ledger to disk after every completed phase.
+  // budget.spent() is an in-memory counter — if the workflow is stopped (manual
+  // stop, crash, timeout) before the Actuals phase, all token data would be lost.
+  // Writing the full ledger here means the file always reflects real consumption
+  // up to the last completed phase, recoverable manually or by a resumed run.
   await agent(
-    `Run these git commands in the repository root:\ngit add -A\ngit commit -m "${commitMsg}"\nIf there is nothing to commit, that is fine — just report success.`,
-    { label: `commit:${implPhase.phase_id}`, phase: 'Implementation', model: 'haiku' }
+    `Write the following JSON to ${featureDir}/${prefix}-token-ledger.json (overwrite the file if it exists):\n${JSON.stringify(tokenLedger, null, 2)}`,
+    { label: `persist-ledger:${implPhase.phase_id}`, phase: 'Implementation', model: 'haiku' }
   )
-
-  if (implPhase.phase_id !== 'INFRA') usPassed.push(implPhase.phase_id)
-  phasesDone++
-  log(`Phase ${implPhase.phase_id} committed`)
 }
 
 // Execute waves: phases in the same wave run in parallel, waves are sequential
@@ -185,57 +260,91 @@ for (const wave of waves) {
   }
 }
 
-log(`Implementation complete — ${phasesDone} phases, ${usPassed.length} US implemented`)
-
-// ── Review — single pass over the full feature diff ───────────────────────────
-phase('Review')
-
-const REVIEW_SCHEMA = {
+// Count open issues
+const ISSUES_SCHEMA = {
   type: 'object',
   properties: {
-    has_critical:  { type: 'boolean' },
-    findings:      { type: 'string' },
-    issues_md:     { type: 'string' },
+    exists:     { type: 'boolean' },
+    open_count: { type: 'number' },
+    path:       { type: 'string' },
   },
-  required: ['has_critical'],
+  required: ['exists', 'open_count'],
 }
+const issuesResult = await agent(
+  `Check if ${wb.feature_dir}/${prefix}-Issues.md exists. If it does, count rows with status OPEN. Return exists, open_count, and the full path.`,
+  { label: 'count-open-issues', phase: 'Implementation', schema: ISSUES_SCHEMA }
+)
+issuesPath = issuesResult?.path || ''
+issuesOpen = issuesResult?.open_count || 0
+log(`Implementation complete — ${phasesDone} phases, ${usPassed.length} US passed, ${usEscalated.length} escalated, ${issuesOpen} open issues`)
+
+// ── Final test run ────────────────────────────────────────────────────────────
+// Developer agents no longer run the test suite themselves (they do a syntax-check
+// build at most). Running the suite once here, after all files are written, replaces
+// N parallel per-agent test runs — cutting ~1h of duplicated builds and artifact
+// contention down to a single centralized run with one consolidated failure report.
+phase('Test')
+
+const beforeTest = budget.spent()
+await agent(
+  `Run the full test suite in the repository root and report the result.
 
-const beforeReview = budget.spent()
-const review = await agent(
-  `${featurePath} --scope all`,
-  {
-    agentType: 'review-solution',
-    label:     'review-solution:full-feature',
-    phase:     'Review',
-    schema:    REVIEW_SCHEMA,
-  }
+1. Read AGENTS.md in the repository root to find the exact test command for this project.
+2. Run that command (e.g. \`dotnet test --no-restore --logger "console;verbosity=normal"\`, \`npm test\`).
+3. Report the outcome: total passed/failed, and for any failures the test name and first error line.
+
+Do NOT fix failing tests — only report them.`,
+  { label: 'final-test-run', phase: 'Test', model: 'haiku' }
 )
-const reviewTokens = budget.spent() - beforeReview
-tokenLedger.push({ agent: 'review-solution:full-feature', model: 'sonnet', phase_delta_tokens: reviewTokens })
+const testTokens = budget.spent() - beforeTest
+tokenLedger.push({ agent: 'final-test-run', model: 'haiku', phase_delta_tokens: testTokens })
+log(`Test run complete — ${testTokens} tokens`)
 
-let issuesOpen     = 0
-let issuesPath     = `${wb.feature_dir}/${prefix}-Issues.md`
-let hasCritical    = false
+// ── Remediation ───────────────────────────────────────────────────────────────
+phase('Remediation')
 
-if (review) {
-  hasCritical = review.has_critical || false
-  if (review.issues_md) {
-    await agent(
-      `Write the following content to ${issuesPath} (create or overwrite the file):\n\n# Issues Register — ${prefix}\n\n${review.issues_md}`,
-      { label: 'write-issues', phase: 'Review', model: 'haiku' }
-    )
-    // Count OPEN issues from issues_md (rough count of "OPEN" occurrences)
-    issuesOpen = (review.issues_md.match(/\bOPEN\b/g) || []).length
-    log(`Review complete — has_critical: ${hasCritical}, open issues: ${issuesOpen}`)
-  } else {
-    log(`Review complete — no issues found`)
-  }
-} else {
-  log(`Review agent returned null — skipping issues register`)
+const REM_SCHEMA = {
+  type: 'object',
+  properties: {
+    issues_fixed:    { type: 'number' },
+    issues_deferred: { type: 'number' },
+  },
+  required: ['issues_fixed', 'issues_deferred'],
 }
 
-if (hasCritical) {
-  log(`WARNING: critical issues found — PR will be created but marked as blocked. Inspect ${prefix}-Issues.md before merging.`)
+let issuesFixed    = 0
+let issuesDeferred = 0
+
+if (issuesOpen > 0 && issuesPath) {
+  log(`Remediation: ${issuesOpen} OPEN issues`)
+  const beforeRem = budget.spent()
+
+  const remResult = await agent(
+    `You are a remediation agent for the feature delivery pipeline.
+
+Read the Issues Register at: ${issuesPath}
+
+For each OPEN issue:
+- If severity is INFO: mark it DEFERRED immediately (update Status column to DEFERRED). Do not dispatch a developer agent.
+- If severity is WARNING: dispatch the appropriate developer agent (developer-backend for most cases) with the issue description. Then spawn review-solution to verify the fix. Max 1 retry per issue. If unresolved after 1 retry, mark DEFERRED.
+
+After processing all issues, return: { "issues_fixed": N, "issues_deferred": N }
+
+Update the Issues Register Status column in place for each resolved/deferred issue.`,
+    {
+      label:  'remediation',
+      phase:  'Remediation',
+      schema: REM_SCHEMA,
+    }
+  )
+
+  const remTokens = budget.spent() - beforeRem
+  tokenLedger.push({ agent: 'remediation', model: 'sonnet', phase_delta_tokens: remTokens })
+  issuesFixed    = remResult.issues_fixed    || 0
+  issuesDeferred = remResult.issues_deferred || 0
+  log(`Remediation done — fixed: ${issuesFixed}, deferred: ${issuesDeferred} — ${remTokens} tokens`)
+} else {
+  log('Remediation: no OPEN issues — skipped')
 }
 
 // ── PR ────────────────────────────────────────────────────────────────────────
@@ -258,37 +367,39 @@ const prResult = await agent(
 Feature path: ${featurePath}
 Branch: ${branch || 'current branch'}
 Prefix: ${prefix}
-Critical issues found: ${hasCritical}
-Open issues: ${issuesOpen}
+Issues fixed: ${issuesFixed}, deferred: ${issuesDeferred}
 
 Your tasks:
 
-1. Push the branch:
+1. If there are remediation fixes, commit them:
+   git add -A
+   git commit -m "fix(${prefix}): remediate review issues"
+
+2. Push the branch:
    git push -u origin ${branch || 'HEAD'}
 
-2. Create a PR targeting develop:
+3. Create a PR targeting develop:
    gh pr create \\
      --title "feat(${prefix}): <Feature Title from feature.md>" \\
      --base develop \\
      --body "## Summary
-- Implements <Feature Title from feature.md> (${usPassed.length} User Stories, ${phasesDone} phases)
+- Implements <Feature Title> (<N> User Stories, <N> tasks)
 - Source docs: ${prefix}-Requirements.md, ${prefix}-Tech-Spec.md
 
 ## Implementation
-- Phases implemented: ${usPassed.join(', ') || 'N/A'}
-- Architect review: ${hasCritical ? '⚠️ CRITICAL issues found — see ' + prefix + '-Issues.md before merging' : '✅ No critical issues'}
-- Open issues: ${issuesOpen} → see ${prefix}-Issues.md
+- Architect review: all US passed
+- Issues Register: ${issuesFixed} fixed, ${issuesDeferred} deferred → see ${prefix}-Issues.md
 
 ## Test plan
 - [ ] Build passes
 - [ ] Tests pass
 - [ ] Manual smoke test of key flows"
 
-3. Update the Feature Registry (internal_docs/features/REGISTRY.md):
-   Find or append the entry for ${prefix}. Set Status to "${hasCritical ? 'review-required' : 'completed'}".
+4. Update the Feature Registry (internal_docs/features/REGISTRY.md):
+   Find or append the entry for ${prefix}. Set Status to "completed".
 
 Return { "pr_url": "<url>", "registry_updated": true/false }.
-If the PR already exists, return its URL. If creation fails, return { "pr_url": "(PR creation failed)", "registry_updated": false }.`,
+If the PR creation fails, return { "pr_url": "(PR creation failed)", "registry_updated": false }.`,
   {
     label:  'pr-and-registry',
     phase:  'PR',
@@ -303,8 +414,41 @@ log(`PR done: ${prResult.pr_url} — ${prTokens} tokens`)
 // ── Actuals ───────────────────────────────────────────────────────────────────
 phase('Actuals')
 
+// Fallback: merge in any ledger entries that were persisted to disk but are missing
+// from the in-memory ledger (e.g. this is a resumed run and earlier phases ran in a
+// previous, interrupted invocation). The in-memory ledger has priority for entries
+// present in both — it reflects the most recent measurements.
+const persistedLedgerRaw = await agent(
+  `Read ${featureDir}/${prefix}-token-ledger.json and return its contents as a raw JSON string, nothing else. If the file does not exist, return exactly "[]".`,
+  { label: 'read-ledger', phase: 'Actuals', model: 'haiku' }
+)
+try {
+  const persisted = JSON.parse(typeof persistedLedgerRaw === 'string' ? persistedLedgerRaw.trim() : '[]')
+  if (Array.isArray(persisted)) {
+    const seen = new Set(tokenLedger.map(e => e.agent))
+    for (const entry of persisted) {
+      if (entry && entry.agent && !seen.has(entry.agent)) {
+        tokenLedger.push(entry)
+        seen.add(entry.agent)
+        log(`Recovered ledger entry from disk: ${entry.agent} (${entry.phase_delta_tokens} tokens)`)
+      }
+    }
+  }
+} catch (e) {
+  log(`Could not parse persisted token ledger — using in-memory ledger only`)
+}
+
 const totalPhase3Tokens = tokenLedger.reduce((s, e) => s + (e.phase_delta_tokens || 0), 0)
 
+const ACTUALS_SCHEMA = {
+  type: 'object',
+  properties: {
+    token_estimate_path:  { type: 'string' },
+    effort_estimate_path: { type: 'string' },
+  },
+  required: [],
+}
+
 const beforeActuals = budget.spent()
 
 await agent(
@@ -348,17 +492,14 @@ ${tokenLedger.map(e => `| ${e.agent} | — | ${e.model} | — | ${e.phase_delta_
 | Metric | Value |
 |--------|-------|
 | Implementation phases done | ${phasesDone} |
-| US implemented | ${usPassed.join(', ') || 'N/A'} |
-| Critical issues | ${hasCritical} |
-| Open issues | ${issuesOpen} |
+| US passed | ${usPassed.join(', ') || 'N/A'} |
+| US escalated | ${usEscalated.join(', ') || 'none'} |
 
 TASK 2 — Update actuals in ${prefix}-Effort-Estimate.md:
-Find the file in the feature directory. Read the existing Per-Phase Breakdown table and update
-the "Actual Agent" column for each phase that was implemented:
-- Phases implemented (${usPassed.join(', ') || 'none'} + INFRA): write "done (agent)"
-- Any phase not in the list: leave "—"
-
-Also append this section at the end of the file:
+Find the file in the feature directory. Read the existing Per-Phase Breakdown table and update the "Actual Agent" column for each phase that was implemented. Use the token ledger to infer which phases completed:
+- Phases in usPassed: ${usPassed.join(', ') || 'none'} — mark Actual Agent as completed (write the wall-clock duration if available, otherwise write "done (agent)")
+- Phases escalated: ${usEscalated.join(', ') || 'none'} — mark Actual Agent as "escalated"
+- Also append this summary section at the end of the file:
 
 ---
 
@@ -367,14 +508,14 @@ Also append this section at the end of the file:
 | Metric | Estimated | Actual |
 |--------|-----------|--------|
 | Implementation phases | ${phasesDone} | ${phasesDone} |
-| Critical review issues | — | ${hasCritical ? 'yes' : 'none'} |
-| Open issues | — | ${issuesOpen} |
+| Issues fixed | — | ${issuesFixed} |
+| Issues deferred | — | ${issuesDeferred} |
 
 Return { "token_estimate_path": "<path>", "effort_estimate_path": "<path>" }.`,
   {
     label:  'write-actuals',
     phase:  'Actuals',
-    schema: { type: 'object', properties: { token_estimate_path: { type: 'string' }, effort_estimate_path: { type: 'string' } }, required: [] },
+    schema: ACTUALS_SCHEMA,
   }
 )
 
@@ -383,22 +524,28 @@ tokenLedger.push({ agent: 'write-actuals', model: 'sonnet', phase_delta_tokens:
 log(`Actuals written — ${actualsTokens} tokens`)
 
 // ── Append to process-log ─────────────────────────────────────────────────────
+const usPassed3    = usPassed.join(', ')    || 'none'
+const usEscalated3 = usEscalated.join(', ') || 'none'
 const phase3Events = [
   `APPROVAL GRANTED by user — Gate 2`,
   `pm-phase3 START — implementation phase — branch: ${branch || '(current)'}`,
   `WB parsed: ${wb.phases.length} phases`,
-  `Implementation DONE — ${phasesDone} phases implemented: ${usPassed.join(', ') || 'none'}`,
-  `Review DONE — critical: ${hasCritical}, open issues: ${issuesOpen}`,
+  `Implementation DONE — ${phasesDone} phases | US passed: ${usPassed3} | US escalated: ${usEscalated3}`,
+  issuesFixed + issuesDeferred > 0
+    ? `Remediation: ${issuesFixed} fixed, ${issuesDeferred} deferred`
+    : `Remediation: skipped (no open issues)`,
   `PR created: ${prResult.pr_url}`,
   `Actuals written: Token-Estimate.md + Effort-Estimate.md`,
   `RUN COMPLETE`,
   `════════════════════════════════════════════════════════`,
 ].join('\n')
 
+const featureDir3 = featurePath.replace(/\/[^/]+$/, '')
+
 await agent(
   `Append phase 3 events to the process-log for this feature delivery run.
 
-Process-log path: ${featureDir}/${prefix}-process-log.txt
+Process-log path: ${featureDir3}/${prefix}-process-log.txt
 
 Steps:
 1. Get current UTC datetime via Bash: run \`date -u +"%Y-%m-%dT%H:%M:%S"\`
@@ -408,14 +555,16 @@ Steps:
 
 Events to append:
 ${phase3Events}`,
-  { label: 'finalize-process-log', phase: 'Actuals', model: 'haiku' }
+  { label: 'finalize-process-log', phase: 'Actuals' }
 )
 
 return {
   pr_url:       prResult.pr_url,
   token_ledger: tokenLedger,
   issues_summary: {
-    has_critical: hasCritical,
-    open_issues:  issuesOpen,
+    escalations:        escalations.length,
+    escalation_details: escalations,
+    issues_fixed:       issuesFixed,
+    issues_deferred:    issuesDeferred,
   },
 }

package/CHANGELOG.md

@@ -1,4 +1,26 @@
 ## 0.1.4 (2026-07-21)
+## [0.7.0](https://github.com/dtlab-tech/ai-toolkit/compare/v0.6.0...v0.7.0) (2026-07-29)
+
+
+### Features
+
+* **pipeline:** centralized test run + persistent token ledger + validation report fix + smarter define-feature ([c9fad6e](https://github.com/dtlab-tech/ai-toolkit/commit/c9fad6e4e5dfe25823df1d6534b4e597baa97001))
+* **pm-phase3:** add centralized test run phase to streamline testing process ([972fac5](https://github.com/dtlab-tech/ai-toolkit/commit/972fac55d0754743859a7d75379a41acd49a890d))
+* **pm-phase3:** persist token ledger to disk after every phase ([b6ad681](https://github.com/dtlab-tech/ai-toolkit/commit/b6ad6814770fae31cda60f25298dab9b2739eeaa))
+
+## [0.6.0](https://github.com/dtlab-tech/ai-toolkit/compare/v0.5.0...v0.6.0) (2026-07-29)
+
+
+### Features
+
+* **define-feature:** context ingestion + adaptive grilling; add AGENTS.md ([581b783](https://github.com/dtlab-tech/ai-toolkit/commit/581b783a18626796ae764ce69f658556b83ab490))
+
+
+### Bug Fixes
+
+* **pipeline:** persist Validation Report + smarter define-feature + AGENTS.md ([a5bae6a](https://github.com/dtlab-tech/ai-toolkit/commit/a5bae6aab65ca78aa5daa08a4a731b7c16051754))
+* **pm-phase1:** guarantee Validation Report is persisted to disk ([53e10c0](https://github.com/dtlab-tech/ai-toolkit/commit/53e10c0e8f20021da9408d6b9e0c077142a449ae))
+
 ## [0.5.0](https://github.com/dtlab-tech/ai-toolkit/compare/v0.4.0...v0.5.0) (2026-07-29)
 
 

package/README.md

@@ -104,7 +104,7 @@ npm run test:coverage  # run all tests and generate a coverage report
 The test suite covers:
 
 - **`tests/cli/`** — unit tests for pure functions in `bin/cli.js`
-  (`fileHash`, `walkDir`, `expandMappings`, `categorize`, `readInstalledVersion`, `isMattPocockInstalled`)
+  (`fileHash`, `walkDir`, `expandMappings`, `categorize`, `readInstalledVersion`)
 - **`tests/frontmatter/`** — structural validation of all `.claude/agents/*.md`
   and `.claude/skills/**/SKILL.md` frontmatter fields
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@dtlabs/ai-toolkit",
-  "version": "0.5.0",
+  "version": "0.7.0",
   "description": "AI Toolkit — reusable agents and procedures for developers to build AI-powered applications.",
   "files": [
     "docs/",