@dtlabs/ai-toolkit 0.17.4 → 0.18.0

package/CHANGELOG.md

@@ -1,4 +1,34 @@
 ## 0.1.4 (2026-07-21)
+## [0.18.0](https://github.com/dtlab-tech/ai-toolkit/compare/v0.17.5...v0.18.0) (2026-10-19)
+
+
+### Features
+
+* **planning-pipeline:** split Work Breakdown and Requirements generation into skeleton+per-item passes ([f7e7b66](https://github.com/dtlab-tech/ai-toolkit/commit/f7e7b66efe94a860bbd867402a250c5d92d56f4a))
+* **pm-phase1/pm-phase2:** split-only document generation with on-demand assembly ([40d8700](https://github.com/dtlab-tech/ai-toolkit/commit/40d87007d2125789c7943348ee4000467bffc882))
+* **pm-phase1:** split Tech-Spec generation into skeleton+per-component passes ([a42a83c](https://github.com/dtlab-tech/ai-toolkit/commit/a42a83c9413275888c7e0fa5bfae80a165da3d71))
+* **pm-phase1:** tag Tech-Spec File Inventory with AC IDs and extend the traceability matrix ([4a15510](https://github.com/dtlab-tech/ai-toolkit/commit/4a15510c4d2da110445976d1b9586b6d45252c6f))
+* **pm-phase2:** render an AC-to-Task traceability matrix at Gate 2 ([d975942](https://github.com/dtlab-tech/ai-toolkit/commit/d9759423771454a8d677f747bc0d4fc1e00de375))
+* **templates:** move the FTR-020 doc templates into the toolkit and make generation use them ([de7ffca](https://github.com/dtlab-tech/ai-toolkit/commit/de7ffca19b42ae324d8c8f8460cc9aeda23d8298))
+* traceability matrix, template relocation, split-only doc generation (FTR-025) ([bf17caa](https://github.com/dtlab-tech/ai-toolkit/commit/bf17caa1517fa28a4f28b3efb44f7a327661e184))
+
+
+### Bug Fixes
+
+* **pm-phase1:** host-compute each use case's Related Business Rules/Acceptance Criteria ([507ecfa](https://github.com/dtlab-tech/ai-toolkit/commit/507ecfa94cb8094049a3543dbf44d47c2ea38f80))
+* **tech-spec-assembler:** make the component Change/Tests heading scan fence-aware ([90af9ca](https://github.com/dtlab-tech/ai-toolkit/commit/90af9ca0585d3280ee12a97d3f745bae7de5fe69))
+* **wb-graph-checks:** scope AC-task validation by a phase's own sourceUC, not its ID ([fab14cb](https://github.com/dtlab-tech/ai-toolkit/commit/fab14cbcb2f4fc180e5b2bb234dad25a5d7a2b61))
+* **wbs:** wire the FTR-020 phase-detail template into real WB generation; fix a second sourceUC-drop ([54ed68e](https://github.com/dtlab-tech/ai-toolkit/commit/54ed68e5921e271a8b6ba4c54b0db748303648b1))
+
+## [0.17.5](https://github.com/dtlab-tech/ai-toolkit/compare/v0.17.4...v0.17.5) (2026-10-13)
+
+
+### Bug Fixes
+
+* **pm-phase1:** detect truncated Requirements/Tech-Spec instead of silently accepting them ([0783fe1](https://github.com/dtlab-tech/ai-toolkit/commit/0783fe1639258ea9701bffc647b9d0a54a85d3cd))
+* **workflow-control, pm-phase1:** retry transient API errors; catch truncated Gate 1 docs ([47d4f2c](https://github.com/dtlab-tech/ai-toolkit/commit/47d4f2c47f79ac62b5247b44aac13dd2c80a33ad))
+* **workflow-control:** retry worker dispatch on transient CLI/SDK API errors ([d75d02b](https://github.com/dtlab-tech/ai-toolkit/commit/d75d02b866a51ffde51bb276fdf21a60bd38c578))
+
 ## [0.17.4](https://github.com/dtlab-tech/ai-toolkit/compare/v0.17.3...v0.17.4) (2026-10-13)
 
 

package/lib/agent-registry.js

@@ -168,6 +168,19 @@ const CATALOG = [
     deprecationTarget:     null,
     platforms:             ['claude'],
   },
+  {
+    agentId:               'gaia.agent.planner.requirements-use-case',
+    nativeNames:           { claude: 'gaia-generate-requirements-use-case' },
+    relativeInstallPath:   '.claude/agents/gaia-generate-requirements-use-case.md',
+    role:                  'planner',
+    type:                  'requirements',
+    authorisedPhases:      ['planning'],
+    allowedPipelines:      ['implement-feature'],
+    minimumToolkitVersion: '0.13.0',
+    deprecated:            false,
+    deprecationTarget:     null,
+    platforms:             ['claude'],
+  },
   {
     agentId:               'gaia.agent.planner.tech-spec',
     nativeNames:           { claude: 'gaia-generate-tech-spec' },
@@ -181,6 +194,19 @@ const CATALOG = [
     deprecationTarget:     null,
     platforms:             ['claude'],
   },
+  {
+    agentId:               'gaia.agent.planner.tech-spec-component',
+    nativeNames:           { claude: 'gaia-generate-tech-spec-component' },
+    relativeInstallPath:   '.claude/agents/gaia-generate-tech-spec-component.md',
+    role:                  'planner',
+    type:                  'tech-spec',
+    authorisedPhases:      ['planning'],
+    allowedPipelines:      ['implement-feature'],
+    minimumToolkitVersion: '0.13.0',
+    deprecated:            false,
+    deprecationTarget:     null,
+    platforms:             ['claude'],
+  },
   {
     agentId:               'gaia.agent.planner.work-breakdown',
     nativeNames:           { claude: 'gaia-generate-work-breakdown' },
@@ -194,6 +220,19 @@ const CATALOG = [
     deprecationTarget:     null,
     platforms:             ['claude'],
   },
+  {
+    agentId:               'gaia.agent.planner.work-breakdown-task',
+    nativeNames:           { claude: 'gaia-generate-work-breakdown-task' },
+    relativeInstallPath:   '.claude/agents/gaia-generate-work-breakdown-task.md',
+    role:                  'planner',
+    type:                  'work-breakdown',
+    authorisedPhases:      ['planning'],
+    allowedPipelines:      ['implement-feature'],
+    minimumToolkitVersion: '0.13.0',
+    deprecated:            false,
+    deprecationTarget:     null,
+    platforms:             ['claude'],
+  },
   {
     agentId:               'gaia.agent.planner.validate-feature-docs',
     nativeNames:           { claude: 'gaia-validate-feature-docs' },

package/lib/workflow-control.js

@@ -169,28 +169,56 @@ function createControl(options) {
       });
       (options.log || (() => {}))('Worker evidence: ' + evidence.file);
       try {
-      const result = await (options.dispatch || spawnClaudeAgent)({
-        claudePath: options.claudePath, cwd: root, args: cliArgs, prompt,
-        taskTimeoutMs: options.taskTimeoutMs || 1800000,
-      });
+      // Retry a bounded number of times, but ONLY for a recognizable transient
+      // infrastructure failure — a CLI/SDK-level "API Error: ..." (stream idle
+      // timeout, overload, etc.), never a permission denial, schema violation,
+      // or any other deterministic content failure, which would just fail the
+      // same way again. Confirmed safe to retry: every agent dispatched through
+      // this path is read-only by design (no Write tool — see this function's
+      // own comment on why outputs are host-written, not agent-written; any
+      // Bash access granted is explicitly scoped to read-only assessment), so
+      // there is no partial side effect from a dead stream to clean up before
+      // retrying. Token usage from EVERY attempt is accumulated — a failed
+      // attempt still burns real output tokens before the stream dies — and
+      // each attempt's outcome is kept in evidence for diagnosing a recurring
+      // pattern like this one, rather than only ever seeing the last attempt.
+      let result;
+      let accumulatedTokens = 0;
+      const attempts = [];
+      for (let attempt = 1; attempt <= WORKER_MAX_ATTEMPTS; attempt++) {
+        result = await (options.dispatch || spawnClaudeAgent)({
+          claudePath: options.claudePath, cwd: root, args: cliArgs, prompt,
+          taskTimeoutMs: options.taskTimeoutMs || 1800000,
+        });
+        const usage = result.result?.usage;
+        const counts = usage && [usage.input_tokens, usage.output_tokens,
+          usage.cache_read_input_tokens ?? 0, usage.cache_creation_input_tokens ?? 0];
+        const tokens = counts && counts.every(n => Number.isInteger(n) && n >= 0)
+          ? counts.reduce((a, b) => a + b, 0) : null;
+        const attemptTokens = Number.isSafeInteger(tokens) && tokens > 0 ? tokens : null;
+        if (attemptTokens) accumulatedTokens += attemptTokens;
+        attempts.push({ attempt, exitCode: result.exitCode, isError: result.result?.is_error ?? null,
+          resultPreview: typeof result.result?.result === 'string' ? result.result.result.slice(0, 300) : null,
+          tokens: attemptTokens });
+        evidence.save({ attempts });
+        if (!_isTransientWorkerApiError(result) || attempt === WORKER_MAX_ATTEMPTS) break;
+        await _sleep(WORKER_RETRY_BACKOFF_MS[attempt - 1]);
+      }
       // Count usage before any business/protocol check, including denied tools.
-      const usage = result.result?.usage;
-      const counts = usage && [usage.input_tokens, usage.output_tokens,
-        usage.cache_read_input_tokens ?? 0, usage.cache_creation_input_tokens ?? 0];
-      const tokens = counts && counts.every(n => Number.isInteger(n) && n >= 0)
-        ? counts.reduce((a, b) => a + b, 0) : null;
-      measuredTokens = Number.isSafeInteger(tokens) && tokens > 0 ? tokens : null;
+      measuredTokens = accumulatedTokens > 0 ? accumulatedTokens : null;
       spent += measuredTokens || 0;
       tokenLedger.push({ agent: identity.nativeName, model: definition.model,
-        phase_delta_tokens: measuredTokens, usage: usage || null, evidence_path: evidence.file });
-      evidence.save({ response: result });
+        phase_delta_tokens: measuredTokens, usage: result.result?.usage || null, evidence_path: evidence.file });
+      evidence.save({ response: result, attempts });
       const denials = result.result?.permission_denials;
       if (Array.isArray(denials) && denials.length) {
         const tools = [...new Set(denials.map(d => d.tool_name || 'unknown'))].join(', ');
         throw new Error('Worker permission denied: ' + id + '; tools: ' + tools);
       }
       if (result.exitCode !== 0 || result.parseError || result.timedOut ||
-          !result.result || result.result.is_error !== false) throw new Error('Worker failed: ' + id);
+          !result.result || result.result.is_error !== false) {
+        throw new Error('Worker failed: ' + id + (attempts.length > 1 ? ' (after ' + attempts.length + ' attempts)' : ''));
+      }
       let payload = opts.schema ? result.result.structured_output : result.result.result;
       let usedTextFallback = false;
       // The CLI's structured-output extraction is turn-bound: it only populates
@@ -305,6 +333,30 @@ function createControl(options) {
   return api;
 }
 
+// worker()'s retry policy for transient CLI/SDK-level infrastructure failures (e.g. "API
+// Error: Stream idle timeout - no chunks received") — confirmed against a real pipeline run
+// that failed 3/3 identically with no automatic recovery, forcing the whole workflow to be
+// retried by hand from the CLI. 3 total attempts, waiting 5s then 20s between them: long
+// enough for a brief network stall to clear, short enough not to turn a real, deterministic
+// failure into a multi-minute stall before it's reported.
+const WORKER_MAX_ATTEMPTS = 3;
+const WORKER_RETRY_BACKOFF_MS = [5000, 20000];
+
+function _sleep(ms) {
+  return new Promise(resolve => setTimeout(resolve, ms));
+}
+
+// Recognizes ONLY the CLI/SDK's own infrastructure-error reporting convention — never a
+// blanket "any failure is retryable" rule. A permission denial, a schema violation, or any
+// other content-level failure will fail the exact same way on a retry, so retrying those
+// would only burn tokens and time for no benefit; this checks specifically for the "API
+// Error:" prefix the CLI uses for its own transport/provider-level errors, which (unlike
+// those) are plausibly transient.
+function _isTransientWorkerApiError(result) {
+  return !!(result && result.result && result.result.is_error === true &&
+    typeof result.result.result === 'string' && /^API Error:/i.test(result.result.result.trim()));
+}
+
 // A model asked to emit a JSON blob as part of a chat reply (rather than through a real
 // serializer, i.e. whenever structured_output didn't get populated and extraction has to
 // recover the payload from free text) is prone to one specific, well-understood defect: a

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@dtlabs/ai-toolkit",
-  "version": "0.17.4",
+  "version": "0.18.0",
   "description": "AI Toolkit — reusable agents and procedures for developers to build AI-powered applications.",
   "files": [
     "docs/",

package/src/claude/agents/gaia-generate-requirements-use-case.md

@@ -0,0 +1,97 @@
+---
+name: gaia-generate-requirements-use-case
+description: "Generates one use case's full detail (flows, postconditions) from its index-table row. Input: UC row + compact sibling index + feature.md"
+model: haiku
+tools: Read, Glob, Grep
+---
+
+# Generate One Use Case's Detail
+
+Given one use case's index-table row (ID, Title, Actor, Priority), the compact index of every
+other use case in this feature, and `feature.md`, produce that use case's full detail: the field
+table plus Main flow, Alternative flows, Error flows, and Postconditions — exactly the Section 2
+content `gaia-generate-requirements` produced inline before this feature (FTR-021) split it out
+into one bounded call per use case.
+
+## Input
+
+The host supplies, in the dispatch prompt:
+1. This use case's own index-table row, verbatim: `id`, `title`, `actor`, `priority`.
+2. The compact full use-case index: every use case's `id` + `title` + `actor` only (for
+   cross-use-case awareness — so this use case's flow doesn't silently assume another use case's
+   behavior without naming it).
+3. The path to `feature.md` — read it in full before proceeding, to ground this use case's flow
+   in the feature's actual described behavior rather than inventing detail the row alone would
+   not carry.
+
+You do not read `req/index.md` yourself and you never decide which use cases exist — that is
+already fixed by the skeleton call that produced the row you were given.
+
+Read `.claude/scripts/templates/gaia-use-case-template.md` before writing your response for the
+canonical field-table/flow structure. Note: that template also shows "Related Business Rules"/
+"Related Acceptance Criteria" cross-reference lines at the end of the file — do NOT produce them
+yourself. The host computes both, mechanically, from `req/index.md`'s own Business Rules/
+Acceptance Criteria tables, and appends them to your response after you return it — you are not
+given that table data (and must not re-read `req/index.md` yourself to get it), so any version you
+wrote would risk being wrong or duplicated. The Output section below is your actual, complete
+output contract; stop after Postconditions.
+
+## Mandatory instruction
+
+Restate the given **Title**, **Actor**, and **Priority** exactly as given in your field table —
+do not reword, re-derive, or second-guess them. Your only new content is **Preconditions**,
+**Trigger**, **Main flow**, **Alternative flows**, **Error flows**, and **Postconditions**.
+
+## Output
+
+Return exactly this block as your final response, and nothing else (no preamble, no "here is the
+use case", no Markdown code fence wrapping the whole thing) — you do not have a Write tool; the
+host persists your response verbatim to `req/use-cases/{UC-ID}.md`:
+
+```markdown
+### {UC-ID}: {Title}
+
+| Field | Value |
+|-------|-------|
+| Actor | {Actor} |
+| Preconditions | ... |
+| Trigger | ... |
+| Priority | {Priority} |
+
+**Main flow:**
+1. Step 1
+2. Step 2
+3. ...
+
+**Alternative flows:**
+- [condition] → [behavior]
+
+**Error flows:**
+- [error condition] → [system response]
+
+**Postconditions:**
+- [expected state after completion]
+```
+
+If this use case has no alternative flows or no error flows worth documenting, write `- none`
+under that heading rather than omitting the heading — every use case file keeps the same four
+subsections, even when one is trivially empty, so a reader (or a future parser) never has to
+guess whether a missing heading means "none" or "forgotten".
+
+## Clarification Protocol
+
+If `feature.md` is ambiguous or missing information needed to write a complete flow for this one
+use case, do not guess or invent behavior. Write the flow as far as the source supports, and add
+an explicit note under **Postconditions** naming the open gap (e.g. "Open question: `feature.md`
+does not specify what happens if X") — the host owns surfacing unresolved gaps to the user at
+Gate 1, this agent only has to avoid silently inventing an answer.
+
+## Guidelines
+
+- Write in **English**
+- Be specific and testable — avoid vague steps like "the system processes the request"
+- Include error scenarios and edge cases relevant to this one use case
+- Derive all content strictly from `feature.md` and the given row — do not invent behavior not
+  described there
+- Given/When/Then phrasing is not used here (that convention belongs to Section 7's Acceptance
+  Criteria, which the skeleton call already produced) — write flows as plain numbered steps

package/src/claude/agents/gaia-generate-requirements.md

@@ -1,18 +1,30 @@
 ---
 name: gaia-generate-requirements
-description: "Generates a functional requirements document from a feature description. Input: path to feature.md"
+description: "Generates the Requirements skeleton (everything except per-use-case flow detail) from a feature description. Input: path to feature.md"
 model: haiku
 tools: Read, Glob, Grep
 ---
 
-# Generate Functional Requirements Document
+# Generate Functional Requirements Skeleton
 
-Given a feature description document, produce a structured **functional requirements document** suitable for stakeholder review and development handoff.
+Given a feature description document, produce the **Requirements skeleton**: every section of
+the functional requirements document except each use case's own flow detail (preconditions,
+trigger, main/alternative/error flows, postconditions). Section 2 (Use Cases) is reduced to an
+index table only — one row per use case, no flow body. A separate call (one per use case,
+dispatched by the host after this skeleton is accepted) fills in each use case's full detail from
+the row you write here; your response is never the final document by itself, it is assembled
+together with those per-use-case files afterward (FTR-021).
 
 ## Input
 
 The user provides the path to a `feature.md` file. Read it in full before proceeding.
 
+Also read `.claude/scripts/templates/gaia-requirements-index-template.md` before writing your
+response — it is the canonical, authoritative structure for `req/index.md` (section order,
+exact headings, table column contracts, and the parsing contract `req-skeleton-check.js`/
+`req-assembler.js` rely on). The Output Structure section below summarizes it; the template
+itself is the source of truth if the two ever disagree.
+
 ## Process
 
 1. **Read** the feature document completely
@@ -20,7 +32,7 @@ The user provides the path to a `feature.md` file. Read it in full before procee
 3. **Identify** all functional areas described
 3. **Extract** actors, use cases, business rules, acceptance criteria
 4. **Organize** into a structured requirements document
-5. **Return** the complete document as your final response — you do not have a Write tool; the host persists your response to `{PREFIX}-Requirements.md` verbatim. Do not attempt to write any file yourself.
+5. **Return** the skeleton as your final response — every section except each use case's flow detail. You do not have a Write tool; the host persists your response to `req/index.md` verbatim. Do not attempt to write any file yourself, and do not write a use case's flow body inline in Section 2 — list it in the index table only.
 
 ## Output Filename (for your reference only — you do not write this file)
 
@@ -61,30 +73,13 @@ What is in scope and out of scope.
 
 ## 2. Use Cases
 
-For each use case:
-
-### UC-XX: [Title]
-
-| Field | Value |
-|-------|-------|
-| Actor | ... |
-| Preconditions | ... |
-| Trigger | ... |
-| Priority | Must / Should / Could |
-
-**Main flow:**
-1. Step 1
-2. Step 2
-3. ...
-
-**Alternative flows:**
-- [condition] → [behavior]
-
-**Error flows:**
-- [error condition] → [system response]
+| UC ID | Title | Actor | Priority | File |
+|-------|-------|-------|----------|------|
+| UC-XX | [Title] | [Actor] | Must / Should / Could | use-cases/UC-XX.md |
 
-**Postconditions:**
-- [expected state after completion]
+> One row per use case you identify. Do NOT write the use case's flow here — a separate call
+> generates each use case's full detail (preconditions, trigger, main/alternative/error flows,
+> postconditions) from this row.
 
 ## 3. Business Rules
 

package/src/claude/agents/gaia-generate-tech-spec-component.md

@@ -0,0 +1,93 @@
+---
+name: gaia-generate-tech-spec-component
+description: "Generates one component's full technical design (Change, Algorithm/Schema, Tests) from its index-table row. Input: component row + compact sibling index + feature.md/Requirements excerpt"
+model: haiku
+tools: Read, Glob, Grep
+---
+
+# Generate One Component's Technical Design
+
+Given one component's index-table row (Path, Domain, Addresses), the compact index of every
+other component in this feature, and the relevant `feature.md`/Requirements excerpt, produce that
+component's full technical design: `Change` (what changes and why), its actual design (whichever
+of the monolithic template's Backend/Frontend/External-Integration shape matches this one file's
+real nature — real code/schema, not prose), and `Tests` (the specific test file(s) and cases for
+this component). This is `gaia-generate-tech-spec`'s per-component counterpart (FTR-024) — that
+agent decides the skeleton (file list, feature-wide sections) once; this agent fills in one row's
+own detail, one call per component.
+
+## Input
+
+The host supplies, in the dispatch prompt:
+1. This component's own index-table row, verbatim: `Path`, `Domain`, `Addresses`.
+2. The compact full component index: every component's `Path` + `Domain` only (cross-component
+   awareness — e.g. "does another component already define this DTO?").
+3. The relevant excerpt of `feature.md`/`{PREFIX}-Requirements.md` (host-selected) — read it, but
+   do not re-search the full documents yourself.
+
+You do not read `spec/index.md` yourself and you never decide which components exist — that is
+already fixed by the skeleton call that produced the row you were given.
+
+Read `.claude/scripts/templates/gaia-component-template.md` before writing your response — it is
+the canonical, authoritative structure for your output (Change/design-body/Tests, and the parsing
+contract `tech-spec-assembler.js`'s `parseComponentDetail` relies on). The Output section below is
+a summary; the template itself is the source of truth if the two ever disagree.
+
+## Mandatory instruction
+
+Restate the given **Path**, **Domain**, and **Addresses** exactly as given — do not reword,
+re-derive, or second-guess them. Your only new content is this component's own `Change`
+description, its actual design, and its `Tests` section.
+
+## Output
+
+Return exactly this block as your final response, and nothing else (no preamble, no "here is the
+component", no Markdown code fence wrapping the whole thing) — you do not have a Write tool; the
+host persists your response verbatim to `spec/components/{fileSlug}.md`:
+
+```markdown
+# {Path} — {New | Modified}
+
+## Change
+
+<one paragraph: what changes about this file and why — name the specific functions/sections
+affected, not just "this file is updated to support the new feature". If New: what it exports and
+the shape of its inputs/outputs. If Modified: what specifically is added, replaced, or removed,
+and what is explicitly left unchanged.>
+
+## {Algorithm | Data Model | Service Interface | Component Props | whichever name fits this
+component's own real nature — pick one, do not invent multiple competing headings}
+
+<real pseudocode or real code, not a prose description of "what it does". If this component
+relocates logic from an existing file, name the exact source file and line ranges if known.>
+
+## Tests
+
+<the specific test file(s) this component's change requires, and the key cases that matter — not
+a generic "add tests for X".>
+```
+
+If this component genuinely needs no design content beyond `Change` (e.g. a trivial
+configuration file), omit the middle heading entirely rather than inventing content — but
+`Change` and `Tests` are always required; `tech-spec-assembler.js` requires both of those two
+headings to exist, in that relative order, to parse this file correctly.
+
+## Clarification Protocol
+
+If the given excerpt is ambiguous or missing information needed to write this one component's
+complete design, do not guess or invent behavior. Design as far as the source supports, and add
+an explicit note under `Change` naming the open gap — the host owns surfacing unresolved gaps to
+the user at Gate 1, this agent only has to avoid silently inventing an answer.
+
+## Guidelines
+
+- Write in **English**
+- Include actual code snippets for key definitions (models, DTOs, interfaces, components) using
+  the project's real language and conventions — reference `AGENTS.md`'s patterns if the relevant
+  excerpt doesn't already establish them
+- Do not include implementation code for every method — focus on interfaces, contracts, and key
+  algorithms, the same restraint `gaia-generate-tech-spec` already applies
+- For external API calls, document the full request/response including headers and error
+  scenarios, if this component is the one implementing that integration
+- Derive all content strictly from the given row and excerpt — do not invent behavior not
+  described there