@dtlabs/ai-toolkit 0.1.7 → 0.2.0

package/.claude/skills/implement-feature/SKILL.md

@@ -5,65 +5,247 @@ argument-hint: <path-to-feature.md> [--force]
 
 # Implement Feature
 
-Orchestrates the full feature delivery pipeline by spawning the `project-manager` agent
-and, once it completes, recording the real token consumption in a dedicated file.
+Orchestrates the full feature delivery pipeline by invoking three sequential workflow phases
+(`pm-phase1`, `pm-phase2`, `pm-phase3`), handling approval gates in the main loop between
+each phase, and recording the real token consumption at the end.
+
+Each workflow phase runs as a real subagent boundary, ensuring worker agents are dispatched
+with their declared `model:` frontmatter honoured and producing accurate per-agent `usage` data.
+
+---
+
+## Step 1 — Invoke pm-phase1 (Documentation Phase)
+
+Invoke the `pm-phase1` workflow with the feature path:
+
+```
+subagent_type: pm-phase1
+prompt: <path-to-feature.md>
+```
+
+Wait for the workflow to complete. Do NOT proceed until it returns.
+
+Extract from the result:
+- `prefix` — feature prefix (e.g. `FTR-009`)
+- `requirements.summary` — requirements file summary
+- `tech_spec.summary` — tech-spec file summary
+- `validation.summary` — validation result (gaps or clean)
+- `token_ledger` — array of per-agent token data from phase 1
+- `errors` — any agent failures during phase 1
+
+If `errors` is non-empty, report them to the user but continue to Gate 1 if the
+required documents were produced.
+
+---
+
+## Step 2 — Present Gate 1 (HARD STOP — present in main loop)
+
+Present the following to the user:
+
+```
+Documents generated for {PREFIX}:
+
+  {PREFIX}-Requirements.md — {requirements.summary}
+  {PREFIX}-Tech-Spec.md    — {tech_spec.summary}
+  {PREFIX}-Validation-Report.md — {validation.summary}
+```
+
+Then output this hard-stop message and **wait for a text reply from the user**:
+
+```
+⛔ GATE 1 — DOCS APPROVAL — HARD STOP
+
+Please review the documents above and reply with one of:
+
+  "Approve — proceed to Work Breakdown"
+  "Request changes: <describe what to change>"
+  "Approve with notes: <your comments>"
+
+The pipeline CANNOT continue until you reply directly.
+```
+
+**If the user requests changes:** note them and stop. Do not invoke pm-phase2.
+
+**If the user approves:** immediately write `{PREFIX}-Approvals.md`:
+
+```markdown
+# Approval Record — {PREFIX}
+
+## Gate 1 — Document Approvals
+
+| Document | Status | Date | Notes |
+|----------|--------|------|-------|
+| {PREFIX}-Requirements.md | ✅ Approved | {today} | {notes or —} |
+| {PREFIX}-Tech-Spec.md | ✅ Approved | {today} | {notes or —} |
+| {PREFIX}-Validation-Report.md | ✅ Approved | {today} | {notes or —} |
+
+## Approval History
+
+| Cycle | Action | Date | Details |
+|-------|--------|------|---------|
+| 1 | Approved | {today} | {user reply text} |
+```
+
+Read back `{PREFIX}-Approvals.md` and verify it exists with Gate 1 ✅ before continuing.
+If the file is missing or incomplete, write it again before proceeding.
+
+---
+
+## Step 3 — Invoke pm-phase2 (Work Breakdown Phase)
+
+Invoke the `pm-phase2` workflow:
+
+```
+subagent_type: pm-phase2
+prompt: <path-to-feature.md>
+```
+
+Wait for the workflow to complete. Extract from the result:
+- `user_stories` — number of User Stories
+- `total_tasks` — total task count
+- `domain_breakdown` — tasks per domain
+- `implementation_phases` — number of phases
+- `human_estimate` — human sequential estimate
+- `agent_estimate` — agent parallel estimate
+- `token_ledger` — per-agent token data from phase 2
 
 ---
 
-## Step 1 — Spawn the Project Manager
+## Step 4 — Present Gate 2 (HARD STOP — present in main loop)
+
+Pre-condition check: read `{PREFIX}-Approvals.md` and verify Gate 1 ✅ is present.
+If missing, return to Step 2.
+
+Present the following to the user:
+
+```
+Work Breakdown for {PREFIX}:
+
+  User Stories: {user_stories} (US-01 ÷ US-{NN})
+  Total tasks:  {total_tasks} ({domain_breakdown})
+  Implementation phases: {implementation_phases}
+  Human estimate: {human_estimate} | Agent estimate: {agent_estimate}
+  Reference: {PREFIX}-Effort-Estimate.md for full detail
+```
+
+Then output this hard-stop message and **wait for a text reply from the user**:
+
+```
+⛔ GATE 2 — WORK BREAKDOWN APPROVAL — HARD STOP
+
+Please review the Work Breakdown above and reply with one of:
+
+  "Approve — start implementation"
+  "Request changes: <describe what to change>"
+  "Approve with notes: <your comments>"
 
-Spawn the `project-manager` agent with the exact arguments the user provided:
+Implementation CANNOT start until you reply directly.
+```
+
+**If the user requests changes:** note them and stop. Do not invoke pm-phase3.
+
+**If the user approves:** append Gate 2 to `{PREFIX}-Approvals.md`:
+
+```markdown
+## Gate 2 — Work Breakdown Approval
 
+| Document | Status | Date | Notes |
+|----------|--------|------|-------|
+| {PREFIX}-Work-Breakdown.md | ✅ Approved | {today} | {total_tasks} tasks, {implementation_phases} phases |
 ```
-subagent_type: project-manager
-prompt: <path-to-feature.md> [--force]
+
+Read back `{PREFIX}-Approvals.md` and verify Gate 2 ✅ is present before continuing.
+
+---
+
+## Step 5 — Create feature branch
+
+Before invoking pm-phase3, create the feature branch:
+
+```bash
+git checkout -b feature/{PREFIX}-{short-slug}
 ```
 
-Wait for the agent to complete. Capture its full result, including the `<usage>` block
-that appears at the end of the tool result (format: `subagent_tokens: N`).
+If the branch already exists, switch to it without recreating:
+
+```bash
+git checkout feature/{PREFIX}-{short-slug}
+```
 
 ---
 
-## Step 2 — Extract orchestrator token usage
+## Step 6 — Invoke pm-phase3 (Implementation Phase)
+
+Pre-condition check: read `{PREFIX}-Approvals.md` and verify BOTH Gate 1 ✅ and Gate 2 ✅.
+If either is missing, return to the missing gate.
+
+Invoke the `pm-phase3` workflow:
 
-From the `<usage>` block of the project-manager result, read:
-- `subagent_tokens` — total tokens consumed by the project-manager agent itself (input + output)
-- `duration_ms` — wall-clock duration of the project-manager run
+```
+subagent_type: pm-phase3
+prompt: <path-to-feature.md> --branch feature/{PREFIX}-{short-slug}
+```
+
+Wait for the workflow to complete. Capture its full result, including the `<usage>` block
+(format: `subagent_tokens: N`). Extract from the result:
+- `pr_url` — the created pull request URL
+- `token_ledger` — all per-agent token data from phase 3
+- `issues_summary` — escalation and issues counts
+
+If `issues_summary.escalations > 0`, report the escalation details to the user.
 
 ---
 
-## Step 3 — Complete Token Estimate file
+## Step 7 — Complete Token Estimate file
+
+From the `<usage>` block of the pm-phase3 result, read:
+- `subagent_tokens` — total tokens consumed by the pm-phase3 workflow
+- `duration_ms` — wall-clock duration of pm-phase3
 
-The project-manager will have already written `{PREFIX}-Token-Estimate.md` with the
-upfront estimate and the child agents' actuals. Read that file, then **append**
-the orchestrator row and the grand total:
+Read `{PREFIX}-Token-Estimate.md`. Append the orchestrator row and grand total:
 
 ```markdown
-| project-manager (orchestrator) | — | sonnet | N (estimated) | N (actual) | ±N | Xh Ymin |
+| project-manager/pm-phase3 (orchestrator) | — | sonnet | 80,000 (estimated) | {subagent_tokens} (actual) | ±{delta} | {duration} |
 
 ---
 
-### Grand Total
+## Actuals vs Estimate
+
+| Agent | Task / Scope | Model | Est. tokens | Actual tokens | Delta | Est. cost ($) | Actual cost ($) | Duration |
+|-------|-------------|-------|------------|---------------|-------|--------------|----------------|----------|
+{one row per entry in token_ledger — from phase 1, 2, and 3}
+| project-manager/pm-phase3 (orchestrator) | — | sonnet | 80,000 | {subagent_tokens} | ±{delta} | $0.4320 | ${actual_cost} | {duration} |
+
+## Estimation accuracy by agent type
+
+| Model | Count | Avg est. tokens | Avg actual tokens | Avg delta | Trend |
+|-------|-------|----------------|------------------|-----------|-------|
+{one row per model tier (haiku, sonnet) — exclude N/A rows from averages}
+
+## Grand Total
 
 | Metric | Estimated | Actual | Delta |
 |--------|-----------|--------|-------|
-| Total tokens (all agents) | N | N | ±N |
-| Total wall-clock | Xh Ymin | Xh Ymin | ±Zmin |
-| Average tokens per agent | N | N | — |
+| Total tokens (all agents) | {sum_est} | {sum_actual} | ±{delta} |
+| Total cost ($) | ${sum_est_cost} | ${sum_actual_cost} | ±${delta_cost} |
+| Total wall-clock | — | {total_duration} | — |
 ```
 
-If the file does not exist yet (project-manager failed before writing it), create it
-from scratch using the data available.
+> Per-agent values marked *(proportional)* are estimated distributions of the phase total.
+> Phase totals and grand total are exact measurements from workflow subagent_tokens.
+
+If the Token Estimate file does not exist (pm-phase3 failed before writing it), create it
+from scratch using the data available in the token_ledger.
 
 ---
 
-## Step 4 — Report to user
+## Step 8 — Report to user
 
 After writing the token file, report:
 
 ```
-✅ Feature pipeline complete.
+Feature pipeline complete.
    Token estimate + actuals → {PREFIX}-Token-Estimate.md
    Effort estimate + actuals → {PREFIX}-Effort-Estimate.md
-   Pull Request → {PR URL}
+   Pull Request → {pr_url}
 ```

package/.claude/workflows/am-phase1.js

@@ -0,0 +1,304 @@
+export const meta = {
+  name: 'am-phase1',
+  description: 'Assessment pipeline phase 1: discover assessment agents → scope filter → parallel assessment → intervention-documentation-standard → Token-Estimate → Effort-Estimate → findings_gate_payload.',
+  phases: [
+    { title: 'Discovery',      detail: 'Scan .claude/agents/ for assessment agents, apply scope filter' },
+    { title: 'Assessment',     detail: 'Dispatch all assessment agents in parallel' },
+    { title: 'Interventions',  detail: 'Run intervention-documentation-standard on all findings' },
+    { title: 'Estimates',      detail: 'Write Token-Estimate and Effort-Estimate files' },
+  ],
+}
+
+// args: "<target_path> [--scope=area1,area2] --prefix ASSESS-NNN [--force]"
+const argStr    = typeof args === 'string' ? args : '.'
+const parts     = argStr.trim().split(/\s+/)
+
+let targetPath  = '.'
+let scope       = null
+let prefix      = null
+let force       = false
+
+for (const part of parts) {
+  if (part.startsWith('--scope='))   { scope  = part.slice('--scope='.length).split(',').map(s => s.trim()) }
+  else if (part === '--force')        { force  = true }
+  else if (part === '--prefix')       { /* next part is value */ }
+  else if (prefix === null && parts[parts.indexOf(part) - 1] === '--prefix') { prefix = part }
+  else if (!part.startsWith('--'))   { targetPath = part }
+}
+
+// Re-parse prefix properly
+const prefixMatch = argStr.match(/--prefix\s+(\S+)/)
+if (prefixMatch) prefix = prefixMatch[1]
+
+if (!prefix) prefix = 'ASSESS-001'
+
+const outputDir = `docs/assessments/${prefix}`
+
+// ── Discovery ─────────────────────────────────────────────────────────────────
+phase('Discovery')
+
+const DISCOVERY_SCHEMA = {
+  type: 'object',
+  properties: {
+    assessment_agents: {
+      type: 'array',
+      items: {
+        type: 'object',
+        properties: {
+          name:        { type: 'string' },
+          description: { type: 'string' },
+        },
+        required: ['name'],
+      },
+    },
+  },
+  required: ['assessment_agents'],
+}
+
+const SCOPE_AGENT_MAP = {
+  architecture:   ['layered-architecture-assessment'],
+  security:       ['security-hardening'],
+  quality:        ['generic-software-assessment'],
+  concurrency:    ['concurrency-safety-assessment'],
+  devops:         [],
+  'domain-model': ['domain-model-refactoring'],
+  dependencies:   ['dependency-supply-chain-security'],
+}
+
+const discovery = await agent(
+  `Scan .claude/agents/ and list all assessment agents.
+
+An assessment agent is a .md file whose name or description contains one of: "assessment", "audit", "analysis".
+Exclude orchestrators (files containing "manager" in their name).
+Exclude remediation-only agents (files containing "refactoring", "hardening", "decomposition" in their name UNLESS they also contain "assessment" or "audit").
+
+For each matching agent, read its frontmatter and extract:
+- name (from the name: field in frontmatter)
+- description (from the description: field)
+
+Return { "assessment_agents": [ { "name": "...", "description": "..." }, ... ] }`,
+  {
+    label:  'discover-assessment-agents',
+    phase:  'Discovery',
+    schema: DISCOVERY_SCHEMA,
+  }
+)
+
+let assessmentAgents = discovery.assessment_agents || []
+
+// Apply scope filter
+if (scope && scope.length > 0) {
+  const allowed = new Set(scope.flatMap(s => SCOPE_AGENT_MAP[s] || []))
+  assessmentAgents = assessmentAgents.filter(a => allowed.has(a.name))
+  log(`Scope filter applied (${scope.join(', ')}): ${assessmentAgents.length} agents selected`)
+} else {
+  log(`No scope filter: ${assessmentAgents.length} assessment agents discovered`)
+}
+
+if (assessmentAgents.length === 0) {
+  log('WARNING: No assessment agents to run — proceeding with empty assessment')
+}
+
+// ── Assessment (parallel) ─────────────────────────────────────────────────────
+phase('Assessment')
+
+log(`Dispatching ${assessmentAgents.length} assessment agents in parallel`)
+
+const tokenLedger   = []
+const beforeAssess  = budget.spent()
+
+const assessmentResults = await parallel(
+  assessmentAgents.map(a => () =>
+    agent(
+      `${targetPath} --prefix ${prefix} --output-dir ${outputDir}`,
+      {
+        agentType: a.name,
+        label:     a.name,
+        phase:     'Assessment',
+      }
+    ).then(result => ({ agent: a.name, success: true,  result }))
+     .catch(err   => ({ agent: a.name, success: false, error: String(err) }))
+  )
+)
+
+const assessTokens = budget.spent() - beforeAssess
+log(`Assessment phase complete — ${assessmentResults.filter(r => r?.success).length}/${assessmentAgents.length} succeeded | total phase delta: ${assessTokens} tokens`)
+
+// Proportional token distribution across agents
+const perAgentTokens = assessmentAgents.length > 0
+  ? Math.round(assessTokens / assessmentAgents.length)
+  : 0
+
+for (const r of assessmentResults.filter(Boolean)) {
+  tokenLedger.push({
+    agent:              r.agent,
+    model:              'sonnet',
+    phase_delta_tokens: perAgentTokens,
+    note:               'proportional distribution of assessment phase total',
+    success:            r.success,
+    error:              r.error || null,
+  })
+}
+
+const failedAgents    = assessmentResults.filter(r => r && !r.success)
+const completedAgents = assessmentResults.filter(r => r &&  r.success)
+
+// ── Interventions ─────────────────────────────────────────────────────────────
+phase('Interventions')
+
+log('Running intervention-documentation-standard')
+
+const beforeInt = budget.spent()
+await agent(
+  `--prefix ${prefix} --output-dir ${outputDir} --target ${targetPath}`,
+  {
+    agentType: 'intervention-documentation-standard',
+    label:     'intervention-documentation-standard',
+    phase:     'Interventions',
+  }
+)
+const intTokens = budget.spent() - beforeInt
+tokenLedger.push({
+  agent:              'intervention-documentation-standard',
+  model:              'sonnet',
+  phase_delta_tokens: intTokens,
+  note:               'exact phase delta',
+})
+log(`intervention-documentation-standard done — phase delta: ${intTokens} tokens`)
+
+// ── Estimates ─────────────────────────────────────────────────────────────────
+phase('Estimates')
+
+const ESTIMATES_SCHEMA = {
+  type: 'object',
+  properties: {
+    severity_counts: {
+      type: 'object',
+      properties: {
+        CRITICAL: { type: 'number' },
+        HIGH:     { type: 'number' },
+        MEDIUM:   { type: 'number' },
+        LOW:      { type: 'number' },
+      },
+    },
+    total_interventions:      { type: 'number' },
+    interventions_index_path: { type: 'string' },
+    token_estimate_path:      { type: 'string' },
+    effort_estimate_path:     { type: 'string' },
+    remediation_hours_est:    { type: 'number' },
+  },
+  required: ['severity_counts', 'total_interventions'],
+}
+
+const totalPhaseTokens = tokenLedger.reduce((s, e) => s + (e.phase_delta_tokens || 0), 0)
+
+const estimatesResult = await agent(
+  `You are the estimates recorder for the assessment pipeline.
+
+Prefix: ${prefix}
+Output directory: ${outputDir}
+Assessment agents that ran: ${assessmentAgents.map(a => a.name).join(', ') || 'none'}
+Failed agents: ${failedAgents.map(r => r.agent).join(', ') || 'none'}
+
+Token ledger (phase-level measurements):
+${JSON.stringify(tokenLedger, null, 2)}
+
+TASK 1 — Read ${prefix}-Interventions-Index.md from ${outputDir}.
+Count rows by Criticality column: CRITICAL, HIGH, MEDIUM, LOW.
+Compute remediation_hours_est = CRITICAL×8 + HIGH×4 + MEDIUM×2 + LOW×1.
+
+TASK 2 — Write Token-Estimate file at ${outputDir}/${prefix}-Token-Estimate.md.
+
+Use this template (fill in all values):
+
+# Token Estimate — ${prefix} — Assessment Pipeline
+
+> Phase totals are exact measurements from budget tracking.
+> Per-agent values within the assessment phase are proportional distributions of the phase total.
+> Pricing model: docs/pricing.md (80% input / 20% output split).
+> ⚠️ Per-agent values marked (proportional) are estimated distributions, not exact measurements.
+
+## Estimation model
+
+| Parameter | Value |
+|-----------|-------|
+| Avg chars per token | 4 |
+| Sonnet system prompt | ~3,000 tokens |
+| Base overhead per call | ~5,000 tokens |
+| Input/output split | 80% / 20% |
+
+## Assessment agents (Phase 1)
+
+| Agent | Model | Phase delta tokens | Note | Status |
+|-------|-------|--------------------|------|--------|
+{one row per agent from token ledger — use phase_delta_tokens and note fields}
+
+## Phase subtotals
+
+| Phase | Tokens (exact) |
+|-------|---------------|
+| Assessment agents (parallel) | {sum of assessment agent phase_delta_tokens} |
+| intervention-documentation-standard | {its phase_delta_tokens} |
+| Total am-phase1 | ${totalPhaseTokens} |
+
+## Grand total
+
+> Updated at pipeline end by assess-codebase skill.
+
+| Metric | Value |
+|--------|-------|
+| Total tokens (am-phase1) | ${totalPhaseTokens} (exact) |
+
+TASK 3 — Write Effort-Estimate file at ${outputDir}/${prefix}-Effort-Estimate.md.
+
+Use this template:
+
+# Effort Estimate — ${prefix} — Assessment Pipeline
+
+> Wall-clock effort tracking for the assessment pipeline.
+> Remediation effort: CRITICAL=8h, HIGH=4h, MEDIUM=2h, LOW=1h (human hours, sequential).
+
+## Assessment phase
+
+| Agent | Status |
+|-------|--------|
+{one row per assessment agent}
+| intervention-documentation-standard | complete |
+
+## Remediation effort estimate
+
+| Severity | Count | Rate | Subtotal |
+|----------|-------|------|---------|
+| CRITICAL | {count} | 8h | {subtotal}h |
+| HIGH | {count} | 4h | {subtotal}h |
+| MEDIUM | {count} | 2h | {subtotal}h |
+| LOW | {count} | 1h | {subtotal}h |
+| **Total** | **{total}** | — | **{total_hours}h** |
+
+Human sequential total: {total_hours}h
+
+Return the structured metrics.`,
+  {
+    label:  'write-estimates',
+    phase:  'Estimates',
+    schema: ESTIMATES_SCHEMA,
+  }
+)
+
+log(`Estimates written — ${estimatesResult.total_interventions} interventions: CRITICAL=${estimatesResult.severity_counts?.CRITICAL} HIGH=${estimatesResult.severity_counts?.HIGH} MEDIUM=${estimatesResult.severity_counts?.MEDIUM} LOW=${estimatesResult.severity_counts?.LOW}`)
+log(`Remediation estimate: ${estimatesResult.remediation_hours_est}h human sequential`)
+
+return {
+  prefix,
+  output_dir:              outputDir,
+  target_path:             targetPath,
+  assessment_summaries:    completedAgents.map(r => ({ agent: r.agent })),
+  interventions_index_path: estimatesResult.interventions_index_path || `${outputDir}/${prefix}-Interventions-Index.md`,
+  severity_counts:         estimatesResult.severity_counts  || { CRITICAL: 0, HIGH: 0, MEDIUM: 0, LOW: 0 },
+  total_interventions:     estimatesResult.total_interventions || 0,
+  remediation_hours_est:   estimatesResult.remediation_hours_est || 0,
+  effort_estimate_path:    estimatesResult.effort_estimate_path || `${outputDir}/${prefix}-Effort-Estimate.md`,
+  token_estimate_path:     estimatesResult.token_estimate_path  || `${outputDir}/${prefix}-Token-Estimate.md`,
+  token_ledger:            tokenLedger,
+  errors:                  failedAgents.map(r => ({ agent: r.agent, error: r.error })),
+}