@dtlabs/ai-toolkit 0.1.7 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/define-feature.md +1 -0
- package/.claude/agents/dependency-injection-refactoring.md +1 -0
- package/.claude/agents/dependency-supply-chain-security.md +1 -0
- package/.claude/agents/developer-backend.md +1 -0
- package/.claude/agents/developer-frontend.md +1 -0
- package/.claude/agents/developer-testing.md +1 -0
- package/.claude/agents/domain-model-refactoring.md +1 -0
- package/.claude/agents/god-class-decomposition.md +1 -0
- package/.claude/agents/init-agents-md.md +1 -0
- package/.claude/agents/install-toolkit.md +134 -3
- package/.claude/agents/intervention-documentation-standard.md +1 -0
- package/.claude/agents/review-solution.md +1 -0
- package/.claude/agents/security-hardening.md +1 -0
- package/.claude/agents/validate-feature-docs.md +2 -0
- package/.claude/settings.json +5 -0
- package/.claude/skills/assess-codebase/SKILL.md +139 -60
- package/.claude/skills/implement-feature/SKILL.md +208 -26
- package/.claude/workflows/am-phase1.js +304 -0
- package/.claude/workflows/am-phase2.js +166 -0
- package/.claude/workflows/pm-phase1.js +164 -0
- package/.claude/workflows/pm-phase2.js +108 -0
- package/.claude/workflows/pm-phase3.js +309 -0
- package/CHANGELOG.md +29 -0
- package/README.md +26 -0
- package/bin/cli.js +64 -5
- package/docs/token-optimization-backlog.md +115 -0
- package/docs/token-optimization.md +274 -0
- package/package.json +1 -1
- package/.claude/agents/assessment-manager.md +0 -672
- package/.claude/agents/project-manager.md +0 -309
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: define-feature
|
|
3
3
|
description: "Define Feature — grills the user with targeted questions to define a new feature, then writes feature.md in docs/features/FTR-XXX-slug/. Output: feature.md ready for /implement-feature"
|
|
4
|
+
model: sonnet
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
# Define Feature
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: dependency-injection-refactoring
|
|
3
3
|
description: "Dependency Injection Refactoring — guides the incremental conversion of tightly coupled or statically accessed dependencies to constructor-injected services, preserving build integrity at every step. Language-agnostic; reads AGENTS.md for framework-specific DI patterns. Input: target classes from intervention document"
|
|
4
|
+
model: sonnet
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
# Dependency Injection Refactoring
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: dependency-supply-chain-security
|
|
3
3
|
description: "Dependency Supply Chain Security — guides the implementation of dependency integrity controls: trusted package sources, deterministic restore via lock files, unused dependency audit, and SCA integration in CI/CD pipelines. Works with npm, pip, NuGet, Maven, and other package managers; reads AGENTS.md for project stack. Output: remediated configuration files + CI/CD pipeline changes"
|
|
4
|
+
model: sonnet
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
# Dependency Supply Chain Security
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: domain-model-refactoring
|
|
3
3
|
description: "Domain Model Refactoring — guides the decomposition of monolithic domain model files, introduction of type hierarchies for structurally similar types, and alignment of model names with domain ubiquitous language. Language-agnostic; reads AGENTS.md for project conventions. Input: target model directory from intervention document"
|
|
4
|
+
model: sonnet
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
# Domain Model Refactoring
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: god-class-decomposition
|
|
3
3
|
description: "God Class Decomposition — guides the safe, incremental decomposition of oversized classes and methods using responsibility cluster analysis, extract-method, extract-class, and parameter object refactoring. Language-agnostic; reads AGENTS.md for project conventions. Input: target class(es) from intervention document"
|
|
4
|
+
model: sonnet
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
# God Class Decomposition
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: init-agents-md
|
|
3
3
|
description: "Analyzes a project codebase and generates a complete AGENTS.md file with tech stack, conventions, real code patterns, build/test commands, and hard constraints. Input: (optional) path to project root — defaults to current working directory."
|
|
4
|
+
model: sonnet
|
|
4
5
|
tools: Read, Glob, Grep, Bash, Write
|
|
5
6
|
---
|
|
6
7
|
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: install-toolkit
|
|
3
3
|
description: "Installs the ai-toolkit into a destination project by copying agents, skills, commands, and procedures. Input: [path-to-destination] [--force] — defaults to current working directory"
|
|
4
|
+
model: sonnet
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
# Install Toolkit
|
|
@@ -43,13 +44,14 @@ The user provides:
|
|
|
43
44
|
|
|
44
45
|
## Step 2 — Plan what to copy
|
|
45
46
|
|
|
46
|
-
|
|
47
|
+
Five source directories to install:
|
|
47
48
|
|
|
48
49
|
| Source (toolkit) | Destination | Purpose |
|
|
49
50
|
|------------------|-------------|---------|
|
|
50
51
|
| `.claude/agents/` | `{dest}/.claude/agents/` | All spawnable subagents |
|
|
51
52
|
| `.claude/skills/` | `{dest}/.claude/skills/` | All user-invocable skills (except `install-toolkit/`) |
|
|
52
53
|
| `.claude/commands/` | `{dest}/.claude/commands/` | All slash commands |
|
|
54
|
+
| `.claude/workflows/` | `{dest}/.claude/workflows/` | Claude Code Workflow scripts for orchestrated pipelines |
|
|
53
55
|
| `docs/procedures/` | `{dest}/docs/procedures/` | Generic procedures (only if destination has no override) |
|
|
54
56
|
|
|
55
57
|
For each file, compare source content against destination using an MD5/hash check and determine status:
|
|
@@ -63,7 +65,6 @@ Build and display the plan before executing:
|
|
|
63
65
|
📦 Install Plan → {destination}
|
|
64
66
|
──────────────────────────────────────────────────
|
|
65
67
|
.claude/agents/
|
|
66
|
-
✅ NEW project-manager.md
|
|
67
68
|
✅ NEW generate-requirements.md
|
|
68
69
|
⚠️ MODIFIED developer-backend.md ← content differs
|
|
69
70
|
⏭ SAME review-solution.md
|
|
@@ -72,6 +73,13 @@ Build and display the plan before executing:
|
|
|
72
73
|
✅ NEW implement-feature/SKILL.md
|
|
73
74
|
⏭ SAME init-agents/SKILL.md
|
|
74
75
|
|
|
76
|
+
.claude/workflows/
|
|
77
|
+
✅ NEW pm-phase1.js
|
|
78
|
+
✅ NEW pm-phase2.js
|
|
79
|
+
✅ NEW pm-phase3.js
|
|
80
|
+
✅ NEW am-phase1.js
|
|
81
|
+
✅ NEW am-phase2.js
|
|
82
|
+
|
|
75
83
|
docs/procedures/
|
|
76
84
|
✅ NEW code-generation.md
|
|
77
85
|
⏭ SAME testing.md
|
|
@@ -106,6 +114,8 @@ cp "{source_file}" "{dest_file}"
|
|
|
106
114
|
|
|
107
115
|
**Never copy the `install-toolkit/` skill directory** — this skill is toolkit-internal and has no use in a destination project.
|
|
108
116
|
|
|
117
|
+
**Never copy `.claude/settings.json` or `.claude/settings.local.json`** — these are user-owned configuration. Copying them would clobber the destination's existing settings. The toolkit's own `settings.json` exists only to configure this repo; it is verified and advised on (Step 4b), never installed.
|
|
118
|
+
|
|
109
119
|
**For `docs/procedures/`**: copy only files that don't already exist in the destination (regardless of `--force`) — project-specific procedure overrides must never be clobbered.
|
|
110
120
|
|
|
111
121
|
---
|
|
@@ -123,6 +133,44 @@ After copying:
|
|
|
123
133
|
|
|
124
134
|
---
|
|
125
135
|
|
|
136
|
+
## Step 4b — Verify subagent spawn depth (verify & advise ONLY — never write)
|
|
137
|
+
|
|
138
|
+
The orchestrated pipelines (`/implement-feature`, `/assess-codebase`) require the
|
|
139
|
+
environment variable `CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH` to be set to `2` or higher.
|
|
140
|
+
Without it (Claude Code default is `1`), an orchestrator running as a subagent cannot
|
|
141
|
+
spawn its worker subagents, so it executes every task inline on its own model —
|
|
142
|
+
per-agent model assignment, context isolation, and per-agent token telemetry are all lost.
|
|
143
|
+
|
|
144
|
+
**This is a check, not a change. Do NOT create or edit any settings file** — merging user
|
|
145
|
+
configuration is out of scope and risks clobbering the user's existing settings.
|
|
146
|
+
|
|
147
|
+
1. Look for the variable in the destination's settings, checking in order:
|
|
148
|
+
- `{dest}/.claude/settings.local.json`
|
|
149
|
+
- `{dest}/.claude/settings.json`
|
|
150
|
+
- `~/.claude/settings.json` (user-global)
|
|
151
|
+
Read each with a tolerant JSON parse; a value of `2` or higher in `env.CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH` satisfies the requirement.
|
|
152
|
+
2. **If satisfied** → report a one-line confirmation and continue.
|
|
153
|
+
3. **If missing / below 2** → print a warning and the exact snippet the user must add themselves:
|
|
154
|
+
```
|
|
155
|
+
⚠️ CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH is not set to 2+.
|
|
156
|
+
Without it, orchestrated pipelines run every worker inline on the
|
|
157
|
+
orchestrator model (per-agent models, context isolation, and token
|
|
158
|
+
telemetry are lost).
|
|
159
|
+
|
|
160
|
+
Add this to .claude/settings.json (this project) or ~/.claude/settings.json
|
|
161
|
+
(all projects) — the installer does NOT edit it for you:
|
|
162
|
+
|
|
163
|
+
{
|
|
164
|
+
"env": {
|
|
165
|
+
"CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH": "2"
|
|
166
|
+
}
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
Then restart Claude Code so the variable is loaded.
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
---
|
|
173
|
+
|
|
126
174
|
## Step 5 — Matt Pocock Skills (version check + optional install/update)
|
|
127
175
|
|
|
128
176
|
After copying the toolkit files, check the state of Matt Pocock's skills:
|
|
@@ -167,7 +215,87 @@ After copying the toolkit files, check the state of Matt Pocock's skills:
|
|
|
167
215
|
|
|
168
216
|
---
|
|
169
217
|
|
|
170
|
-
## Step 6 —
|
|
218
|
+
## Step 6 — Compact Instructions opt-in
|
|
219
|
+
|
|
220
|
+
The toolkit includes a `# Compact instructions` section for `~/.claude/CLAUDE.md` that guides Claude's auto-compaction (what to keep, what to drop) and enables proactive topic-change suggestions.
|
|
221
|
+
|
|
222
|
+
1. **Offer the opt-in** — inform the user and ask:
|
|
223
|
+
|
|
224
|
+
> "Would you like to add a `# Compact instructions` section to your global `~/.claude/CLAUDE.md`?
|
|
225
|
+
> It guides auto-compaction to preserve decision-critical information and suggests running `/compact` when you switch topics."
|
|
226
|
+
>
|
|
227
|
+
> Options: "Yes — add it" / "No — skip"
|
|
228
|
+
|
|
229
|
+
2. If the user selects **No**: skip silently; set `compact_instructions_status = "skipped (user said No)"`.
|
|
230
|
+
|
|
231
|
+
3. If the user selects **Yes**:
|
|
232
|
+
|
|
233
|
+
a. **Resolve path**: `CLAUDE_MD="$HOME/.claude/CLAUDE.md"` (Windows bash: `$USERPROFILE/.claude/CLAUDE.md` if `$HOME` is not set).
|
|
234
|
+
|
|
235
|
+
b. **Idempotency check**: search the file for the exact heading `# Compact instructions`:
|
|
236
|
+
```bash
|
|
237
|
+
grep -q "^# Compact instructions" "$CLAUDE_MD" 2>/dev/null && echo "exists" || echo "absent"
|
|
238
|
+
```
|
|
239
|
+
- If **exists**: display skip notice and set `compact_instructions_status = "skipped (already present)"`. Do **not** modify the file.
|
|
240
|
+
- If **absent** (or file does not exist): continue to step (c).
|
|
241
|
+
|
|
242
|
+
c. **Request confirmation** — display exactly:
|
|
243
|
+
> "I am about to append a `# Compact instructions` section to your global `~/.claude/CLAUDE.md`. Confirm to proceed."
|
|
244
|
+
|
|
245
|
+
- If the user **declines**: skip silently; set `compact_instructions_status = "skipped (declined)"`.
|
|
246
|
+
- If the user **confirms**: continue to step (d).
|
|
247
|
+
|
|
248
|
+
d. **Write the section** — append verbatim to `~/.claude/CLAUDE.md` (create the file if it does not exist):
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
mkdir -p "$(dirname "$CLAUDE_MD")"
|
|
252
|
+
cat >> "$CLAUDE_MD" << 'COMPACT_SECTION'
|
|
253
|
+
|
|
254
|
+
# Compact instructions
|
|
255
|
+
|
|
256
|
+
## What to preserve
|
|
257
|
+
When compacting, always keep:
|
|
258
|
+
- Current objective and active task
|
|
259
|
+
- Confirmed decisions and user approvals
|
|
260
|
+
- User answers to questions asked during the session
|
|
261
|
+
- Paths of files created or modified
|
|
262
|
+
- Open errors and unresolved blockers
|
|
263
|
+
- Finding IDs, artifact IDs, FTR/ASSESS/INT reference numbers
|
|
264
|
+
|
|
265
|
+
## What to discard
|
|
266
|
+
When compacting, drop:
|
|
267
|
+
- Raw grep and search results
|
|
268
|
+
- Successful tool outputs (file reads, bash commands that completed without error)
|
|
269
|
+
- Repeated or superseded explanations
|
|
270
|
+
- Superseded plans and intermediate reasoning steps
|
|
271
|
+
|
|
272
|
+
## Topic-change notification
|
|
273
|
+
When the user sends a message containing any of the following phrases (case-insensitive):
|
|
274
|
+
"passiamo a", "ora facciamo", "prossimo punto", "prossimo argomento", "cambiamo argomento",
|
|
275
|
+
"nuovo argomento", "let's move on to", "next topic", "switching to", "now let's do",
|
|
276
|
+
"moving on to", "next up" —
|
|
277
|
+
|
|
278
|
+
— pause before answering and send exactly this message:
|
|
279
|
+
|
|
280
|
+
> **Context compaction suggested**
|
|
281
|
+
>
|
|
282
|
+
> You are about to switch to a new topic. Compacting the context now will keep the next session clean and focused.
|
|
283
|
+
>
|
|
284
|
+
> Reply **Approve - Compact context** to run `/compact` now, or continue to proceed without compacting.
|
|
285
|
+
|
|
286
|
+
If the user replies "Approve - Compact context", run `/compact`. Otherwise proceed normally.
|
|
287
|
+
COMPACT_SECTION
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
Report success: "✅ `# Compact instructions` section added to `~/.claude/CLAUDE.md`."
|
|
291
|
+
Set `compact_instructions_status = "✅ added to ~/.claude/CLAUDE.md"`.
|
|
292
|
+
|
|
293
|
+
**Skip notice** (displayed when section already exists):
|
|
294
|
+
> "A `# Compact instructions` section already exists in `~/.claude/CLAUDE.md`. No changes were made. Review and update it manually if needed."
|
|
295
|
+
|
|
296
|
+
---
|
|
297
|
+
|
|
298
|
+
## Step 7 — Report
|
|
171
299
|
|
|
172
300
|
```
|
|
173
301
|
✅ Toolkit installed at {destination}
|
|
@@ -177,6 +305,8 @@ Modified/overwritten: N
|
|
|
177
305
|
Modified/kept: N (user chose to keep existing version)
|
|
178
306
|
Unchanged (same): N (skipped silently)
|
|
179
307
|
Matt Pocock skills: ✅ installed v{X} / ⬆️ updated v{old}→v{new} / ⏭ skipped / ✅ already up to date v{X}
|
|
308
|
+
Spawn depth: ✅ CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH=2+ present / ⚠️ MISSING — see action above
|
|
309
|
+
Compact instructions: {compact_instructions_status}
|
|
180
310
|
──────────────────────────────────────────────────
|
|
181
311
|
Next steps:
|
|
182
312
|
1. Run /init-agents to generate AGENTS.md for this project
|
|
@@ -192,5 +322,6 @@ If any modified files were kept by the user (not overwritten), list them so they
|
|
|
192
322
|
|
|
193
323
|
- **Never overwrite `docs/procedures/` files** regardless of `--force` — project overrides take priority
|
|
194
324
|
- **Never copy `install-toolkit.md`** — it's a toolkit-internal utility
|
|
325
|
+
- **Never copy or edit `settings.json` / `settings.local.json`** — user-owned config; only verify and advise (Step 4b)
|
|
195
326
|
- **Create directories as needed** — destination may not have `.claude/` yet
|
|
196
327
|
- **Abort cleanly** if destination path is invalid — do not create partial installs
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: intervention-documentation-standard
|
|
3
3
|
description: "Intervention Documentation Standard — generates structured, self-contained intervention documents from assessment findings. Each document is independently actionable by an architect or developer agent. Input: assessment findings; Output: {ASSESS_PREFIX}-INT-NNN-{slug}.md per finding + Interventions-Index.md"
|
|
4
|
+
model: sonnet
|
|
4
5
|
tools: Read, Grep, Glob, Bash, Write
|
|
5
6
|
---
|
|
6
7
|
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: review-solution
|
|
3
3
|
description: "Senior Architect — full solution review across quality, reuse, architecture, and security dimensions. Reports findings without auto-fixing. Input: scope (files, US-XX, or git diff)"
|
|
4
|
+
model: opus
|
|
4
5
|
tools: Read, Grep, Glob, Bash
|
|
5
6
|
---
|
|
6
7
|
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: security-hardening
|
|
3
3
|
description: "Security Hardening — guides the implementation of targeted security fixes: input validation at system boundaries, parameterised queries, secret management, log sanitisation, and TLS/transport configuration. Language-agnostic; reads AGENTS.md for project-specific patterns. Input: security findings from assessment"
|
|
4
|
+
model: sonnet
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
# Security Hardening
|
|
@@ -123,6 +123,8 @@ For each document with gaps, invoke the corresponding agent with a **targeted re
|
|
|
123
123
|
|
|
124
124
|
## Phase 6 — Write Validation Report
|
|
125
125
|
|
|
126
|
+
**Always execute this phase**, regardless of whether gaps were found. The report is required even when validation is clean.
|
|
127
|
+
|
|
126
128
|
Write `{PREFIX}-Validation-Report.md` in the same directory:
|
|
127
129
|
|
|
128
130
|
```markdown
|
|
@@ -5,66 +5,150 @@ argument-hint: "[path] [--scope=architecture,security,quality,concurrency,devops
|
|
|
5
5
|
|
|
6
6
|
# Assess Codebase
|
|
7
7
|
|
|
8
|
-
Orchestrates the full codebase assessment pipeline by
|
|
9
|
-
|
|
8
|
+
Orchestrates the full codebase assessment pipeline by invoking two sequential workflow phases
|
|
9
|
+
(`am-phase1`, `am-phase2`), handling the Findings Gate in the main loop between the phases,
|
|
10
|
+
and recording the real token consumption at the end.
|
|
11
|
+
|
|
12
|
+
Each workflow phase runs as a real subagent boundary, ensuring assessment agents are dispatched
|
|
13
|
+
with accurate per-agent `usage` data and their declared `model:` frontmatter honoured.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Step 1 — Determine assessment prefix
|
|
18
|
+
|
|
19
|
+
Scan `docs/assessments/` for folders matching `ASSESS-[0-9]+*`. Increment the highest number
|
|
20
|
+
found, or start at `ASSESS-001`. This prefix is used for all output files in this run.
|
|
10
21
|
|
|
11
22
|
---
|
|
12
23
|
|
|
13
|
-
## Step
|
|
24
|
+
## Step 2 — Invoke am-phase1 (Assessment Phase)
|
|
14
25
|
|
|
15
|
-
|
|
26
|
+
Invoke the `am-phase1` workflow:
|
|
16
27
|
|
|
17
28
|
```
|
|
18
|
-
subagent_type:
|
|
19
|
-
prompt: <path> [--scope
|
|
29
|
+
subagent_type: am-phase1
|
|
30
|
+
prompt: <path> [--scope=<areas>] --prefix ASSESS-NNN [--force]
|
|
20
31
|
```
|
|
21
32
|
|
|
22
|
-
If no path
|
|
33
|
+
If no path was provided by the user, use `.` (current working directory).
|
|
34
|
+
Always pass `--prefix ASSESS-NNN` with the prefix determined in Step 1.
|
|
35
|
+
|
|
36
|
+
Wait for the workflow to complete. Capture its full result, including the `<usage>` block
|
|
37
|
+
(format: `subagent_tokens: N`). Extract from the result:
|
|
38
|
+
- `prefix` — confirmed assessment prefix
|
|
39
|
+
- `output_dir` — path to `docs/assessments/ASSESS-NNN/`
|
|
40
|
+
- `assessment_summaries` — list of `{ agent, output_file }` for each completed assessment
|
|
41
|
+
- `interventions_index_path` — path to `{PREFIX}-Interventions-Index.md`
|
|
42
|
+
- `severity_counts` — `{ CRITICAL, HIGH, MEDIUM, LOW }`
|
|
43
|
+
- `total_interventions` — total intervention count
|
|
44
|
+
- `remediation_hours_est` — estimated remediation hours
|
|
45
|
+
- `effort_estimate_path` — path to `{PREFIX}-Effort-Estimate.md`
|
|
46
|
+
- `token_estimate_path` — path to `{PREFIX}-Token-Estimate.md`
|
|
47
|
+
- `errors` — any agent failures during phase 1
|
|
48
|
+
|
|
49
|
+
If `errors` is non-empty, report them to the user before presenting the Findings Gate.
|
|
50
|
+
Partial assessments are acceptable — present the Findings Gate with available data.
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Step 3 — Present Findings Gate (HARD STOP — present in main loop)
|
|
55
|
+
|
|
56
|
+
The Findings Gate has two mandatory steps.
|
|
57
|
+
|
|
58
|
+
### Step 3a — Acknowledge
|
|
59
|
+
|
|
60
|
+
Present the findings summary to the user:
|
|
61
|
+
|
|
62
|
+
```
|
|
63
|
+
Assessment complete for {prefix} — {output_dir}
|
|
64
|
+
────────────────────────────────────────────────────────────
|
|
65
|
+
Findings: {CRITICAL} CRITICAL | {HIGH} HIGH | {MEDIUM} MEDIUM | {LOW} LOW
|
|
66
|
+
Interventions: {total_interventions} proposed
|
|
67
|
+
────────────────────────────────────────────────────────────
|
|
68
|
+
Estimated remediation effort: {remediation_hours_est}h (human sequential)
|
|
69
|
+
Reference: {effort_estimate_path} for full breakdown
|
|
70
|
+
────────────────────────────────────────────────────────────
|
|
71
|
+
{list each assessment agent and its output file}
|
|
72
|
+
────────────────────────────────────────────────────────────
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Then output this hard-stop message and **wait for any non-empty text reply from the user**:
|
|
76
|
+
|
|
77
|
+
```
|
|
78
|
+
⛔ FINDINGS GATE — ASSESSMENT ACKNOWLEDGEMENT — HARD STOP
|
|
79
|
+
|
|
80
|
+
Please review the assessment findings above.
|
|
81
|
+
|
|
82
|
+
Reply with any text to acknowledge (e.g. "Acknowledged", "OK", "Proceed").
|
|
83
|
+
You will then be asked which interventions to flag for feature delivery.
|
|
84
|
+
|
|
85
|
+
The pipeline CANNOT continue until you reply directly.
|
|
86
|
+
```
|
|
23
87
|
|
|
24
|
-
|
|
25
|
-
that appears at the end of the tool result (format: `subagent_tokens: N`).
|
|
88
|
+
Capture the acknowledgement text.
|
|
26
89
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
90
|
+
### Step 3b — Flag interventions
|
|
91
|
+
|
|
92
|
+
After acknowledgement, read `{PREFIX}-Interventions-Index.md` and list all INT-NNN identifiers.
|
|
93
|
+
Present them to the user and prompt:
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
Which interventions do you want to flag for feature delivery?
|
|
97
|
+
|
|
98
|
+
{list of INT-NNN — title — criticality}
|
|
99
|
+
|
|
100
|
+
Reply with a comma-separated list of INT-NNN identifiers (e.g. "INT-001, INT-003"),
|
|
101
|
+
or reply "None" to flag nothing.
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
Wait for a text reply. Validate each supplied identifier against the Interventions Index.
|
|
105
|
+
If any are unknown, list them and re-prompt once. Accept "None" for zero flagged.
|
|
106
|
+
|
|
107
|
+
Capture the flagged identifiers (as a comma-separated string, or "none").
|
|
30
108
|
|
|
31
109
|
---
|
|
32
110
|
|
|
33
|
-
## Step
|
|
111
|
+
## Step 4 — Invoke am-phase2 (Approvals and Registry Phase)
|
|
34
112
|
|
|
35
|
-
|
|
36
|
-
- `subagent_tokens` — total tokens consumed by the assessment-manager agent itself (input + output)
|
|
37
|
-
- `duration_ms` — wall-clock duration of the assessment-manager run
|
|
113
|
+
Invoke the `am-phase2` workflow:
|
|
38
114
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
115
|
+
```
|
|
116
|
+
subagent_type: am-phase2
|
|
117
|
+
prompt: --prefix {prefix} --output-dir {output_dir} --flagged {flagged_ids} --ack "{acknowledgement_text}"
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Where `{flagged_ids}` is the comma-separated list of INT-NNN identifiers (or "none").
|
|
121
|
+
|
|
122
|
+
Wait for the workflow to complete. Extract from the result:
|
|
123
|
+
- `approvals_path` — path to `{PREFIX}-Approvals.md`
|
|
124
|
+
- `registry_updated` — registry write result string
|
|
125
|
+
- `summary` — full assessment summary text
|
|
42
126
|
|
|
43
127
|
---
|
|
44
128
|
|
|
45
|
-
## Step
|
|
129
|
+
## Step 5 — Complete Token Estimate file
|
|
46
130
|
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
131
|
+
From the `<usage>` block of the am-phase1 result, read:
|
|
132
|
+
- `subagent_tokens` — total tokens consumed by the am-phase1 workflow
|
|
133
|
+
- `duration_ms` — wall-clock duration of am-phase1
|
|
50
134
|
|
|
51
|
-
|
|
135
|
+
If the `<usage>` block is missing: log warning
|
|
136
|
+
`"am-phase1 produced no <usage> block; orchestrator token data unavailable"`
|
|
137
|
+
and set `actual_tokens = "N/A"`.
|
|
52
138
|
|
|
53
|
-
|
|
139
|
+
Read `{token_estimate_path}`. Append the following in order:
|
|
54
140
|
|
|
55
|
-
|
|
141
|
+
### 5a — Orchestrator row (for use in 5b only)
|
|
56
142
|
|
|
143
|
+
Compute values:
|
|
57
144
|
- Estimated tokens: 80,000 (baseline from estimation model)
|
|
58
|
-
- Estimated cost: $0.4320 —
|
|
59
|
-
|
|
60
|
-
- Actual
|
|
61
|
-
- Actual cost: `(actual_tokens / 1,000) × blended_rate_per_1k`, formatted to 4 decimal places; `"N/A"` if actual_tokens is `"N/A"` or pricing data is unavailable
|
|
145
|
+
- Estimated cost: $0.4320 — `80,000 × $0.005400 / 1,000` using sonnet blended rate
|
|
146
|
+
- Actual tokens: `subagent_tokens` from `<usage>` block; `"N/A"` if missing
|
|
147
|
+
- Actual cost: `(actual_tokens / 1,000) × 0.005400` at 4 decimal places; `"N/A"` if unavailable
|
|
62
148
|
|
|
63
|
-
###
|
|
149
|
+
### 5b — Actuals vs Estimate section
|
|
64
150
|
|
|
65
|
-
Append a horizontal rule (`---`) followed by
|
|
66
|
-
row for every agent that ran: all assessment agents, intervention-documentation-standard,
|
|
67
|
-
and the assessment-manager orchestrator.
|
|
151
|
+
Append a horizontal rule (`---`) followed by:
|
|
68
152
|
|
|
69
153
|
```markdown
|
|
70
154
|
## Actuals vs Estimate
|
|
@@ -72,19 +156,18 @@ and the assessment-manager orchestrator.
|
|
|
72
156
|
| Agent | Task/Scope | Model | Est. tokens | Actual tokens | Delta | Est. cost ($) | Actual cost ($) | Duration |
|
|
73
157
|
|-------|------------|-------|-------------|---------------|-------|---------------|-----------------|----------|
|
|
74
158
|
| {agent} | {scope} | {model} | {est} | {actual} | {±delta} | {est_cost} | {actual_cost} | {Xmin Ys} |
|
|
159
|
+
...
|
|
160
|
+
| assessment-manager/am-phase1 (orchestrator) | — | sonnet | 80,000 | {actual} | ±{delta} | $0.4320 | ${actual_cost} | {duration} |
|
|
75
161
|
```
|
|
76
162
|
|
|
77
163
|
Column rules:
|
|
78
|
-
- **Delta**: `actual_tokens − est_tokens`; show as `+N` or `−N`. Show `"N/A"` if either
|
|
79
|
-
value is missing.
|
|
164
|
+
- **Delta**: `actual_tokens − est_tokens`; show as `+N` or `−N`. Show `"N/A"` if either missing.
|
|
80
165
|
- **Duration**: convert `duration_ms` to `Xmin Ys` where available; use `"—"` where not.
|
|
81
|
-
- Rows with `"N/A"` actual tokens are still shown
|
|
82
|
-
aggregate calculations.
|
|
166
|
+
- Rows with `"N/A"` actual tokens are still shown but excluded from aggregate calculations.
|
|
83
167
|
|
|
84
|
-
###
|
|
168
|
+
### 5c — Estimation accuracy by agent type (conditional)
|
|
85
169
|
|
|
86
|
-
If 2
|
|
87
|
-
(e.g., haiku AND sonnet), append:
|
|
170
|
+
If 2+ distinct model tiers appear with non-N/A actuals, append:
|
|
88
171
|
|
|
89
172
|
```markdown
|
|
90
173
|
## Estimation accuracy by agent type
|
|
@@ -94,16 +177,13 @@ If 2 or more distinct model tiers appear in the rows that have non-`"N/A"` actua
|
|
|
94
177
|
| {model} | {N} | {avg_est} | {avg_actual} | {avg_delta} | {trend} |
|
|
95
178
|
```
|
|
96
179
|
|
|
97
|
-
Column rules:
|
|
98
180
|
- One row per model tier (e.g., `haiku`, `sonnet`)
|
|
99
|
-
- Exclude
|
|
100
|
-
- **Trend**: `"over-target"` if avg delta > +20% of avg est; `"under-target"` if
|
|
101
|
-
< −20% of avg est; otherwise `"on-target"`
|
|
181
|
+
- Exclude N/A rows from averages
|
|
182
|
+
- **Trend**: `"over-target"` if avg delta > +20% of avg est; `"under-target"` if < −20%; else `"on-target"`
|
|
102
183
|
|
|
103
|
-
###
|
|
184
|
+
### 5d — Grand Total section
|
|
104
185
|
|
|
105
|
-
Update the Grand Total section
|
|
106
|
-
`"partial — updated at pipeline end"` marker with `"Final"` and fill in the computed values:
|
|
186
|
+
Update the Grand Total section: replace `"partial — updated at pipeline end"` with Final values:
|
|
107
187
|
|
|
108
188
|
```markdown
|
|
109
189
|
## Grand Total (Final)
|
|
@@ -115,27 +195,26 @@ Update the Grand Total section already written in the file: replace the
|
|
|
115
195
|
| Total wall-clock | — | {Xmin Ys} | — | — |
|
|
116
196
|
```
|
|
117
197
|
|
|
118
|
-
|
|
119
|
-
- **Delta %**: `(delta / estimated) × 100` at 1 decimal place; show `"N/A"` if estimated is 0
|
|
120
|
-
|
|
121
|
-
Summation rules:
|
|
198
|
+
- **Delta %**: `(delta / estimated) × 100` at 1 decimal place; `"N/A"` if estimated is 0
|
|
122
199
|
- Sum all agents' tokens and costs; exclude `"N/A"` rows from sums
|
|
123
|
-
- Cost
|
|
124
|
-
- The "Remediation" section
|
|
125
|
-
- Wall-clock: convert `duration_ms` from
|
|
200
|
+
- Cost: 2 decimal places for totals; 4 decimal places for per-row costs
|
|
201
|
+
- The "Remediation" section contains a static note — leave it as-is
|
|
202
|
+
- Wall-clock: convert `duration_ms` from am-phase1 `<usage>` to minutes/seconds
|
|
126
203
|
|
|
127
204
|
---
|
|
128
205
|
|
|
129
|
-
## Step
|
|
206
|
+
## Step 6 — Report to user
|
|
130
207
|
|
|
131
208
|
After writing to the Token Estimate file, report:
|
|
132
209
|
|
|
133
210
|
```
|
|
134
211
|
Assessment pipeline complete.
|
|
135
|
-
Token estimate + actuals → {
|
|
136
|
-
Approvals →
|
|
137
|
-
Process log →
|
|
212
|
+
Token estimate + actuals → {token_estimate_path}
|
|
213
|
+
Approvals → {approvals_path}
|
|
214
|
+
Process log → {output_dir}/{prefix}-process-log.txt
|
|
215
|
+
|
|
216
|
+
{summary from am-phase2}
|
|
138
217
|
```
|
|
139
218
|
|
|
140
|
-
If the Token Estimate file does not exist (
|
|
219
|
+
If the Token Estimate file does not exist (am-phase1 failed before writing it):
|
|
141
220
|
note this in the report but do not halt — the skill has completed its work.
|