@tyroneross/build-loop 0.35.0 → 0.36.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/marketplace.json +1 -1
- package/.claude-plugin/marketplace.json +3 -3
- package/.claude-plugin/plugin.json +2 -2
- package/.codex-plugin/plugin.json +1 -1
- package/AGENTS.md +67 -14
- package/README.md +208 -299
- package/agents/advisor.md +2 -0
- package/agents/alignment-checker.md +2 -0
- package/agents/api-assessor.md +3 -1
- package/agents/architecture-scout.md +2 -0
- package/agents/assessment-orchestrator.md +2 -0
- package/agents/build-orchestrator.md +13 -8
- package/agents/database-assessor.md +107 -3
- package/agents/design-contract-specialist.md +2 -0
- package/agents/fact-checker.md +2 -0
- package/agents/fix-critique.md +26 -5
- package/agents/frontend-assessor.md +3 -1
- package/agents/implementer.md +2 -0
- package/agents/independent-auditor.md +2 -0
- package/agents/mock-scanner.md +2 -0
- package/agents/optimize-runner.md +2 -0
- package/agents/overfitting-reviewer.md +2 -0
- package/agents/performance-assessor.md +3 -1
- package/agents/plan-critic.md +2 -0
- package/agents/promotion-reviewer.md +2 -0
- package/agents/recurring-pattern-detector.md +4 -1
- package/agents/retrospective-synthesizer.md +14 -0
- package/agents/root-cause-investigator.md +39 -2
- package/agents/scope-auditor.md +2 -0
- package/agents/security-reviewer.md +3 -0
- package/agents/self-improvement-architect.md +2 -0
- package/agents/synthesis-critic.md +2 -0
- package/agents/transcript-pattern-miner.md +2 -0
- package/agents/ui-validator.md +2 -0
- package/assets/build-loop-plugin-icon.png +0 -0
- package/bin/build-loop-install.js +234 -0
- package/commands/run.md +1 -1
- package/hooks/git/pre-commit +40 -0
- package/hooks/git/pre-push +35 -10
- package/hooks/hooks.json +42 -5
- package/hooks/pre-edit-architecture.sh +2 -2
- package/hooks/route-guard.sh +15 -0
- package/hooks/route_guard.py +221 -0
- package/hooks/session-start-extensions.sh +10 -0
- package/hooks/session-start-git-hooks.sh +6 -1
- package/hooks/session-start-rally-point.sh +23 -1
- package/hooks/test_session_start_rally_version.sh +123 -0
- package/package.json +7 -2
- package/plugin-artifacts/codex/.codex-plugin/plugin.json +1 -1
- package/plugin-artifacts/codex/AGENTS.md +67 -14
- package/plugin-artifacts/codex/README.md +208 -299
- package/plugin-artifacts/codex/assets/build-loop-plugin-icon.png +0 -0
- package/plugin-artifacts/codex/references/backlog-system.md +285 -0
- package/plugin-artifacts/codex/references/capability-routing.md +49 -0
- package/plugin-artifacts/codex/references/coordination-rules.md +271 -2
- package/plugin-artifacts/codex/references/memory-systems.md +25 -2
- package/plugin-artifacts/codex/references/memory.md +14 -0
- package/plugin-artifacts/codex/references/model-tier-mapping.md +45 -6
- package/plugin-artifacts/codex/references/phase-1-assess.md +19 -1
- package/plugin-artifacts/codex/references/phase-2-plan.md +9 -1
- package/plugin-artifacts/codex/references/phase-3-execute.md +1 -1
- package/plugin-artifacts/codex/references/phase-4-review.md +20 -1
- package/plugin-artifacts/codex/references/phase-5-iterate.md +4 -3
- package/plugin-artifacts/codex/references/phase-6-learn.md +2 -0
- package/plugin-artifacts/codex/references/verify-dispatch.md +85 -0
- package/plugin-artifacts/codex/skills/build-loop/SKILL.md +19 -10
- package/plugin-artifacts/codex/skills/build-loop/fallbacks.md +1 -1
- package/plugin-artifacts/codex/skills/build-loop/references/capability-routing.md +49 -0
- package/plugin-artifacts/codex/skills/build-loop/references/experiment-results-template.md +101 -0
- package/plugin-artifacts/codex/skills/build-loop/references/memory.md +14 -0
- package/plugin-artifacts/codex/skills/build-loop/references/output-style.md +98 -5
- package/plugin-artifacts/codex/skills/build-loop/references/phase-1-assess.md +19 -1
- package/plugin-artifacts/codex/skills/build-loop/references/phase-2-plan.md +9 -1
- package/plugin-artifacts/codex/skills/build-loop/references/phase-3-execute.md +1 -1
- package/plugin-artifacts/codex/skills/build-loop/references/phase-4-review.md +20 -1
- package/plugin-artifacts/codex/skills/build-loop/references/phase-5-iterate.md +4 -3
- package/plugin-artifacts/codex/skills/build-loop/references/phase-6-learn.md +2 -0
- package/plugin-artifacts/codex/skills/build-loop/references/verify-dispatch.md +85 -0
- package/scripts/build_codex_plugin_artifact.py +7 -0
- package/scripts/embed_backend.py +17 -13
- package/scripts/lessons_index/query.py +1 -1
- package/scripts/memory_graph/__init__.py +2 -2
- package/scripts/sync_navgator_lessons.py +4 -4
- package/skills/api-registry-bridge/SKILL.md +1 -1
- package/skills/auto-decision-capture/SKILL.md +4 -3
- package/skills/auto-finding-capture/SKILL.md +119 -0
- package/skills/build-loop/SKILL.md +19 -10
- package/skills/build-loop/fallbacks.md +1 -1
- package/skills/build-loop/references/capability-routing.md +49 -0
- package/skills/build-loop/references/experiment-results-template.md +101 -0
- package/skills/build-loop/references/memory.md +14 -0
- package/skills/build-loop/references/output-style.md +98 -5
- package/skills/build-loop/references/phase-1-assess.md +19 -1
- package/skills/build-loop/references/phase-2-plan.md +9 -1
- package/skills/build-loop/references/phase-3-execute.md +1 -1
- package/skills/build-loop/references/phase-4-review.md +20 -1
- package/skills/build-loop/references/phase-5-iterate.md +4 -3
- package/skills/build-loop/references/phase-6-learn.md +2 -0
- package/skills/build-loop/references/verify-dispatch.md +85 -0
- package/skills/debug-loop/SKILL.md +4 -2
- package/skills/debugging-memory/SKILL.md +24 -12
- package/skills/{debugging/assess/SKILL.md → debugging-memory/references/assess.md} +6 -13
- package/skills/{debugging/memory/SKILL.md → debugging-memory/references/search.md} +10 -17
- package/skills/{debugging/store/SKILL.md → debugging-memory/references/store.md} +5 -13
- package/skills/debugging-memory/references/subagent-integration.md +3 -3
- package/skills/defenseclaw-bridge/SKILL.md +1 -1
- package/skills/{loop-builder → focused-loop-builder}/SKILL.md +6 -6
- package/skills/ibr-bridge/SKILL.md +1 -1
- package/skills/knowledge/SKILL.md +13 -2
- package/skills/{knowledge-review/SKILL.md → knowledge/references/review-mode.md} +8 -20
- package/skills/logging-tracer/SKILL.md +20 -4
- package/skills/model-bakeoff/SKILL.md +44 -0
- package/skills/model-tiering/SKILL.md +55 -20
- package/skills/optimize/SKILL.md +2 -2
- package/skills/prd-bridge/SKILL.md +1 -1
- package/skills/recursive-retrospective/SKILL.md +46 -0
- package/skills/recursive-retrospective/references/01-retrospective.md +136 -0
- package/skills/recursive-retrospective/references/02-learning-capture.md +72 -0
- package/skills/recursive-retrospective/references/03-judge.md +45 -0
- package/skills/research/SKILL.md +1 -1
- package/skills/root-cause-analysis/SKILL.md +42 -0
- package/skills/security-methodology/SKILL.md +10 -1
- package/skills/security-scan/SKILL.md +55 -0
- package/skills/spec-writing/SKILL.md +42 -1
- package/skills/sync-skills/SKILL.md +8 -7
- package/commands/assess.md +0 -41
- package/commands/compose-handoff.md +0 -113
- package/commands/debug.md +0 -20
- package/commands/debugger-detail.md +0 -34
- package/commands/debugger-scan.md +0 -15
- package/commands/debugger-status.md +0 -20
- package/commands/debugger.md +0 -32
- package/commands/optimize-run.md +0 -31
- package/commands/promote-experiment.md +0 -129
- package/commands/rally-point.md +0 -301
- package/commands/research-run.md +0 -22
- package/commands/review-knowledge.md +0 -16
- package/commands/self-review.md +0 -51
- package/commands/setup-memory.md +0 -32
- package/commands/start-prd.md +0 -57
- package/commands/test.md +0 -34
- package/commands/verify-plan.md +0 -39
- package/skills/debugging/debug-loop/SKILL.md +0 -231
- package/skills/logging-tracer-bridge/SKILL.md +0 -70
- /package/skills/{loop-builder → focused-loop-builder}/presets/active-project-evidence.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/presets/generic-artifact-loop.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/presets/presentation-audit.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/presets/research-synthesis.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/presets/source-ingestion-raw-data-audit.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/references/spec-format.md +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/scripts/loop_builder.py +0 -0
|
@@ -1,11 +1,4 @@
|
|
|
1
|
-
|
|
2
|
-
name: build-loop:debugging-memory-search
|
|
3
|
-
description: Memory-first lookup before debugging — search local build-loop incidents and optionally standalone Coding Debugger memory. Build-loop's native debugging memory, adapted from debugger workflows. Distinct from the legacy in-tree `build-loop:debugging-memory` skill (kept for backward compat); this is the source-tracked native version.
|
|
4
|
-
version: 0.1.0
|
|
5
|
-
user-invocable: false
|
|
6
|
-
source: claude-code-debugger/skills/debugging-memory/SKILL.md
|
|
7
|
-
source_hash: 5c4ee5ada781107e7def92abeca4d51fc0efc61700f7cf43e948da34f4c0681d
|
|
8
|
-
---
|
|
1
|
+
<!-- PROVENANCE: op=search reference for `build-loop:debugging-memory` (ADR-01 op-routing). Folded from skills/debugging/memory/SKILL.md (former skill name build-loop:debugging-memory-search, v0.1.0) on 2026-07-02, pool-consolidation Inc 5. Drift-check vs upstream retired (native, adapted; no canonical upstream). Former provenance for record: source=claude-code-debugger/skills/debugging-memory/SKILL.md source_hash=5c4ee5ada781107e7def92abeca4d51fc0efc61700f7cf43e948da34f4c0681d -->
|
|
9
2
|
|
|
10
3
|
<!-- SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com> | SPDX-License-Identifier: Apache-2.0 -->
|
|
11
4
|
|
|
@@ -13,7 +6,7 @@ source_hash: 5c4ee5ada781107e7def92abeca4d51fc0efc61700f7cf43e948da34f4c0681d
|
|
|
13
6
|
|
|
14
7
|
Memory-first debugging. Core principle: **never solve the same bug twice**. Native to build-loop; initially adapted from the debugger workflow lineage. Search local `.build-loop/issues/` first, then use standalone Coding Debugger for cross-project memory only when that plugin is installed.
|
|
15
8
|
|
|
16
|
-
> **
|
|
9
|
+
> **Op-routing note**: this is the `op: "search"` reference for `build-loop:debugging-memory` — the memory LOOKUP step. Callers invoke `Skill("build-loop:debugging-memory") with input { op: "search", symptom, domain? }`; this file holds the lookup procedure the workflow delegates to (ADR-01).
|
|
17
10
|
|
|
18
11
|
## When to Activate
|
|
19
12
|
|
|
@@ -32,7 +25,7 @@ The search returns a **verdict** with matching incidents and patterns when struc
|
|
|
32
25
|
1. **KNOWN_FIX**: Apply the documented fix directly only when the strict direct-apply gate (below) passes; otherwise adapt the prior incident as a hypothesis and route to the standard fix flow
|
|
33
26
|
2. **LIKELY_MATCH**: Review the past incident, use it as a starting point — never direct-apply
|
|
34
27
|
3. **WEAK_SIGNAL**: Consider loosely related incidents, but investigate fresh
|
|
35
|
-
4. **NO_MATCH**: Proceed with standard debugging via `build-loop:
|
|
28
|
+
4. **NO_MATCH**: Proceed with standard debugging via `build-loop:debug-loop`, then document the solution after
|
|
36
29
|
|
|
37
30
|
### Direct-apply gate (KNOWN_FIX only)
|
|
38
31
|
|
|
@@ -59,7 +52,7 @@ When this skill activates, always announce it to the user:
|
|
|
59
52
|
|
|
60
53
|
## Deep Investigation Mode
|
|
61
54
|
|
|
62
|
-
For non-trivial issues, escalate to the `build-loop:
|
|
55
|
+
For non-trivial issues, escalate to the `build-loop:debug-loop` skill. Trigger is the **verdict category**, not a numeric confidence score:
|
|
63
56
|
|
|
64
57
|
- **`KNOWN_FIX`** → apply directly, skip the loop
|
|
65
58
|
- **`LIKELY_MATCH`** → enter debug loop (past incidents need verification against current context)
|
|
@@ -82,7 +75,7 @@ Also enter the debug loop when:
|
|
|
82
75
|
|
|
83
76
|
## Incident Documentation
|
|
84
77
|
|
|
85
|
-
After fixing a bug, store via `build-loop:debugging-store`. Required fields: `symptom`, `root_cause`, `fix`. Optional: `category`, `tags`, `files_changed`, `file`.
|
|
78
|
+
After fixing a bug, store via `build-loop:debugging-memory` `{op:"store"}`. Required fields: `symptom`, `root_cause`, `fix`. Optional: `category`, `tags`, `files_changed`, `file`.
|
|
86
79
|
|
|
87
80
|
## Quality Indicators
|
|
88
81
|
|
|
@@ -122,7 +115,7 @@ Use these only when standalone Coding Debugger is installed. Build-loop does not
|
|
|
122
115
|
|
|
123
116
|
Closes the memory-first gate's feedback loop. Both required:
|
|
124
117
|
|
|
125
|
-
- For each newly resolved Review-B/Iterate failure: invoke `build-loop:debugging-store` with `{symptom, root_cause, fix, tags: ["build-loop", project, layer], files}`
|
|
118
|
+
- For each newly resolved Review-B/Iterate failure: invoke `build-loop:debugging-memory` `{op:"store"}` with `{symptom, root_cause, fix, tags: ["build-loop", project, layer], files}`
|
|
126
119
|
- For each Review-B memory gate where standalone Coding Debugger supplied a prior `KNOWN_FIX` or `LIKELY_MATCH`: invoke its `outcome` tool with `{incident_id, result: "worked"|"failed"|"modified", notes}` — this trains the optional verdict classifier
|
|
127
120
|
|
|
128
121
|
Skipping `outcome` means the optional verdict classifier never improves.
|
|
@@ -134,14 +127,14 @@ When debugging involves subagents:
|
|
|
134
127
|
1. **Pre-query memory once** through `build-loop:debugging-memory` before spawning agents
|
|
135
128
|
2. **Distribute context** — each agent gets relevant subset
|
|
136
129
|
3. **Aggregate findings** — collect insights from all agents
|
|
137
|
-
4. **Store unified incident** — single `build-loop:debugging-store` call to document combined diagnosis
|
|
130
|
+
4. **Store unified incident** — single `build-loop:debugging-memory` `{op:"store"}` call to document combined diagnosis
|
|
138
131
|
|
|
139
132
|
Subagents do not inherit Skill or MCP access — pre-load context into their prompt.
|
|
140
133
|
|
|
141
134
|
## Sibling Skills
|
|
142
135
|
|
|
143
|
-
- `build-loop:debugging-store` — write incident after fix
|
|
144
|
-
- `build-loop:debugging-assess` — parallel domain assessment for multi-domain symptoms
|
|
145
|
-
- `build-loop:
|
|
136
|
+
- `build-loop:debugging-memory` `{op:"store"}` — write incident after fix
|
|
137
|
+
- `build-loop:debugging-memory` `{op:"assess"}` — parallel domain assessment for multi-domain symptoms
|
|
138
|
+
- `build-loop:debug-loop` — iterative root-cause analysis with causal-tree investigation
|
|
146
139
|
|
|
147
140
|
*Source: adapted from the debugger workflow lineage and maintained as a build-loop-native skill. Drift-checked by `build-loop:sync-skills`.*
|
|
@@ -1,12 +1,4 @@
|
|
|
1
|
-
|
|
2
|
-
name: build-loop:debugging-store
|
|
3
|
-
description: Store a debugging incident after fixing a bug — writes build-loop's native local incident record and optionally mirrors to standalone Coding Debugger. Build-loop's native incident storage; canonical source has no discrete SKILL.md (the §"Incident Documentation" section of debugging-memory).
|
|
4
|
-
version: 0.1.0
|
|
5
|
-
user-invocable: false
|
|
6
|
-
source: claude-code-debugger/skills/debugging-memory/SKILL.md
|
|
7
|
-
source_hash: 5c4ee5ada781107e7def92abeca4d51fc0efc61700f7cf43e948da34f4c0681d
|
|
8
|
-
source_section: "Incident Documentation"
|
|
9
|
-
---
|
|
1
|
+
<!-- PROVENANCE: op=store reference for `build-loop:debugging-memory` (ADR-01 op-routing). Folded from skills/debugging/store/SKILL.md (former skill name build-loop:debugging-store, v0.1.0) on 2026-07-02, pool-consolidation Inc 5. Drift-check vs upstream retired (native, adapted; no canonical upstream). Former provenance for record: source=claude-code-debugger/skills/debugging-memory/SKILL.md §Incident-Documentation source_hash=5c4ee5ada781107e7def92abeca4d51fc0efc61700f7cf43e948da34f4c0681d -->
|
|
10
2
|
|
|
11
3
|
<!-- SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com> | SPDX-License-Identifier: Apache-2.0 -->
|
|
12
4
|
|
|
@@ -19,7 +11,7 @@ Persist a fixed bug to debugging memory so future builds can recognize recurrenc
|
|
|
19
11
|
## When to Activate
|
|
20
12
|
|
|
21
13
|
- Phase 4 Review-F Report: for each Review-B/Iterate failure newly resolved this build, store the incident
|
|
22
|
-
- After any `build-loop:
|
|
14
|
+
- After any `build-loop:debug-loop` run that produced a verified fix
|
|
23
15
|
- User asks "save this fix", "remember this bug"
|
|
24
16
|
|
|
25
17
|
## Native Path — Build-Loop Incident Note
|
|
@@ -146,8 +138,8 @@ Write to `.build-loop/debugging/incidents/<incident_id>.json`. Flag `debugger JS
|
|
|
146
138
|
|
|
147
139
|
## Sibling Skills
|
|
148
140
|
|
|
149
|
-
- `build-loop:debugging-memory` — search memory before debugging
|
|
150
|
-
- `build-loop:debugging-assess` — parallel domain assessment
|
|
151
|
-
- `build-loop:
|
|
141
|
+
- `build-loop:debugging-memory` `{op:"search"}` (references/search.md) — search memory before debugging
|
|
142
|
+
- `build-loop:debugging-memory` `{op:"assess"}` (references/assess.md) — parallel domain assessment
|
|
143
|
+
- `build-loop:debug-loop` — full iterative debugging that produces the incident this op stores
|
|
152
144
|
|
|
153
145
|
*Source: adapted from the debugger incident-documentation workflow and maintained as a build-loop-native skill. Drift-checked by `build-loop:sync-skills`.*
|
|
@@ -55,7 +55,7 @@ This project uses build-loop native debugging memory.
|
|
|
55
55
|
|
|
56
56
|
**Before investigating any bug:**
|
|
57
57
|
\`\`\`
|
|
58
|
-
Skill("build-loop:debugging-memory
|
|
58
|
+
Skill("build-loop:debugging-memory") with input { op: "search", symptom: "<symptom description>" }
|
|
59
59
|
\`\`\`
|
|
60
60
|
|
|
61
61
|
**Interpret results:**
|
|
@@ -88,10 +88,10 @@ When subagents cannot directly access debugging memory (no Bash tool, sandboxed,
|
|
|
88
88
|
|
|
89
89
|
**Step 1: Search before spawning**
|
|
90
90
|
|
|
91
|
-
Use `build-loop:debugging-memory
|
|
91
|
+
Use `build-loop:debugging-memory` `{op:"search"}` with the symptom description. If the subagent cannot invoke skills, the parent reads `.build-loop/issues/` and passes compact matches:
|
|
92
92
|
|
|
93
93
|
```
|
|
94
|
-
Skill("build-loop:debugging-memory
|
|
94
|
+
Skill("build-loop:debugging-memory") with input { op: "search", symptom: "user login failing with 401" }
|
|
95
95
|
```
|
|
96
96
|
|
|
97
97
|
**Step 2: Format context for subagent**
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: build-loop:defenseclaw-bridge
|
|
3
|
-
description: Use when the user is working on the defenseclaw project and build-loop's Phase 1 detects defenseclaw-specific files (CLAUDE.md indicates the bridge target). Bridges build-loop's orchestrator to defenseclaw's eval/test conventions; otherwise no-op.
|
|
3
|
+
description: Use when the user is working on the defenseclaw project and build-loop's Phase 1 detects defenseclaw-specific files (CLAUDE.md indicates the bridge target). Bridges build-loop's orchestrator to defenseclaw's eval/test conventions; otherwise no-op. NOT a general eval/test bridge (defenseclaw project only); for API/PRD/UI use `api-registry-bridge` / `prd-bridge` / `ibr-bridge`.
|
|
4
4
|
version: 0.1.0
|
|
5
5
|
user-invocable: false
|
|
6
6
|
---
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
name: loop-builder
|
|
2
|
+
name: focused-loop-builder
|
|
3
3
|
description: Use when the user asks to "create a custom build loop", "build a loop spec", "make a focused loop", "generate a workflow loop", "adapt a framework into a loop", or asks whether a workflow should use skill chaining. Generates declarative focused-loop specs, presets, validators, and skill-chain plans.
|
|
4
4
|
version: 0.1.0
|
|
5
5
|
user-invocable: false
|
|
@@ -7,7 +7,7 @@ user-invocable: false
|
|
|
7
7
|
|
|
8
8
|
<!-- SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com> | SPDX-License-Identifier: Apache-2.0 -->
|
|
9
9
|
|
|
10
|
-
# loop-builder
|
|
10
|
+
# focused-loop-builder
|
|
11
11
|
|
|
12
12
|
Create declarative focused-loop specs that reuse build-loop's phase discipline outside pure coding work. The skill turns a preset or framework into a small loop pack: `loop.yaml`, `rubric.md`, report template, validator stub, and explicit skill-chain guidance.
|
|
13
13
|
|
|
@@ -37,7 +37,7 @@ Keep the runner generic and the loop domain-specific. Do not create a new orches
|
|
|
37
37
|
4. **Generate the loop pack.**
|
|
38
38
|
- Run:
|
|
39
39
|
```bash
|
|
40
|
-
python3 skills/loop-builder/scripts/loop_builder.py create <loop-id> --preset <preset-name>
|
|
40
|
+
python3 skills/focused-loop-builder/scripts/loop_builder.py create <loop-id> --preset <preset-name>
|
|
41
41
|
```
|
|
42
42
|
- Default output is `.build-loop/loops/<loop-id>/`.
|
|
43
43
|
- Use `--output <dir>` for a WorkWiki, ObsidianVault, or non-code project workspace.
|
|
@@ -76,19 +76,19 @@ Avoid chaining when:
|
|
|
76
76
|
List presets:
|
|
77
77
|
|
|
78
78
|
```bash
|
|
79
|
-
python3 skills/loop-builder/scripts/loop_builder.py list
|
|
79
|
+
python3 skills/focused-loop-builder/scripts/loop_builder.py list
|
|
80
80
|
```
|
|
81
81
|
|
|
82
82
|
Create a loop:
|
|
83
83
|
|
|
84
84
|
```bash
|
|
85
|
-
python3 skills/loop-builder/scripts/loop_builder.py create active-project-evidence --preset active-project-evidence
|
|
85
|
+
python3 skills/focused-loop-builder/scripts/loop_builder.py create active-project-evidence --preset active-project-evidence
|
|
86
86
|
```
|
|
87
87
|
|
|
88
88
|
Inspect a preset:
|
|
89
89
|
|
|
90
90
|
```bash
|
|
91
|
-
python3 skills/loop-builder/scripts/loop_builder.py inspect presentation-audit
|
|
91
|
+
python3 skills/focused-loop-builder/scripts/loop_builder.py inspect presentation-audit
|
|
92
92
|
```
|
|
93
93
|
|
|
94
94
|
## Additional Resources
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: build-loop:ibr-bridge
|
|
3
|
-
description: Routing bridge to the IBR plugin for UI visual verification. Build-loop prefers IBR `scan` / `scan_macos` when the IBR plugin is installed; otherwise falls back to build-loop's own `native-ax-driver` / `ui-validator`. Never falls back to nm/strings.
|
|
3
|
+
description: Routing bridge to the IBR plugin for UI visual verification. Build-loop prefers IBR `scan` / `scan_macos` when the IBR plugin is installed; otherwise falls back to build-loop's own `native-ax-driver` / `ui-validator`. Never falls back to nm/strings. NOT for API registration (use `api-registry-bridge`) or PRD grounding (use `prd-bridge`).
|
|
4
4
|
version: 0.3.0
|
|
5
5
|
user-invocable: false
|
|
6
6
|
---
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: knowledge
|
|
3
|
-
description: Canonical build-loop-memory framework. Use when the user asks to "record a decision", "log an ADR", "write an MADR", "capture this choice", "regenerate the decisions index", "validate knowledge", "migrate feedback to decisions", or "recall <topic>". Active durable writes go to `~/dev/git-folder/build-loop-memory`; legacy `.episodic/` paths are migration/archive inputs only.
|
|
3
|
+
description: Canonical build-loop-memory framework. Use when the user asks to "record a decision", "log an ADR", "write an MADR", "capture this choice", "regenerate the decisions index", "validate knowledge", "migrate feedback to decisions", or "recall <topic>". ALSO the read-only review surface (review mode): "review my decisions", "show review queue", "check decision rot", "list open conflicts", "find stale procedures", or `/knowledge:review`. Active durable writes go to `~/dev/git-folder/build-loop-memory`; legacy `.episodic/` paths are migration/archive inputs only.
|
|
4
4
|
user-invocable: true
|
|
5
5
|
when_to_use: |
|
|
6
6
|
- User wants to record a substantive choice with rationale
|
|
@@ -114,12 +114,23 @@ postgresql://tyroneross@localhost:5432/agent_memory). Per-project schema:
|
|
|
114
114
|
this repo uses `build_loop_memory`. The schema name is configurable via
|
|
115
115
|
the `--schema` flag on each DB-aware script.
|
|
116
116
|
|
|
117
|
+
## Review mode (read-only)
|
|
118
|
+
|
|
119
|
+
The read-only review surface — `/knowledge:review` and asks like "review my
|
|
120
|
+
decisions", "show review queue", "check decision rot", "list open conflicts",
|
|
121
|
+
"find stale procedures" — lists the four sections of decisions/procedures
|
|
122
|
+
awaiting human attention (review queue, decision rot, open conflicts, stale
|
|
123
|
+
procedures) with a suggested action per item. It NEVER auto-resolves; humans
|
|
124
|
+
take the action. Full surface, invocation flags, and the consolidation
|
|
125
|
+
cross-reference: **`references/review-mode.md`**. (Namespace note: this surface
|
|
126
|
+
reviews the legacy `.episodic/` paths; active durable writes target
|
|
127
|
+
build-loop-memory canonical.)
|
|
128
|
+
|
|
117
129
|
## What's NOT in this skill
|
|
118
130
|
|
|
119
131
|
- Auto-capture from conversation — Phase 3 (`auto-decision-capture`
|
|
120
132
|
skill, Stop hook with `scan_transcript_for_decisions.py`)
|
|
121
133
|
- Memory consolidation — Phase 4 (`consolidate_memory.py`)
|
|
122
|
-
- `/knowledge:review` slash command — Phase 4
|
|
123
134
|
- `derived/libraries.json` and `derived/CHANGELOG.md` generators —
|
|
124
135
|
Phase 1.5 / 4
|
|
125
136
|
|
|
@@ -1,24 +1,7 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: knowledge-review
|
|
3
|
-
description: Repo-local episodic memory review surface. Use when the user asks to "review my decisions", "show review queue", "check decision rot", "list open conflicts", "find stale procedures", or runs `/knowledge:review`. Read-only — never auto-resolves.
|
|
4
|
-
user-invocable: false
|
|
5
|
-
when_to_use: |
|
|
6
|
-
- User runs `/knowledge:review` or asks to surface review-needing items
|
|
7
|
-
- User wants to see decisions older than the staleness threshold
|
|
8
|
-
- User wants to see `_review/` queue items awaiting promotion
|
|
9
|
-
- User wants to see open `fact_conflicts` rows
|
|
10
|
-
- User wants to see procedures whose `depends_on` symbols are missing from the codebase
|
|
11
|
-
namespace: .episodic/decisions/_review/, .episodic/decisions/, .procedural/, agent_memory.<schema>.fact_conflicts
|
|
12
|
-
companion_scripts:
|
|
13
|
-
- scripts/knowledge_review.py — aggregates all four sections into a markdown report
|
|
14
|
-
- scripts/detect_decision_rot.py — drives the rot section
|
|
15
|
-
- scripts/procedural_governance.py — drives the stale-procedures section (validate-symbols mode)
|
|
16
|
-
- scripts/consolidate_memory.py — referenced as the next-step action when surface items accumulate
|
|
17
|
-
---
|
|
18
|
-
|
|
19
1
|
<!-- SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com> | SPDX-License-Identifier: Apache-2.0 -->
|
|
2
|
+
<!-- PROVENANCE: folded from skills/knowledge-review/SKILL.md (v0.1.0, user-invocable:false) on 2026-07-02 (pool-consolidation Inc 3). Review-intent selection triggers preserved in skills/knowledge/SKILL.md frontmatter description (W4); this reference holds the review-surface body. Namespace note: this surface reviews the LEGACY .episodic/ paths (the knowledge skill's active writes target build-loop-memory canonical); .episodic is migration/archive input. -->
|
|
20
3
|
|
|
21
|
-
#
|
|
4
|
+
# Knowledge review mode — surface review-needing items
|
|
22
5
|
|
|
23
6
|
`/knowledge:review` is the read-only review surface for the four-memory-types
|
|
24
7
|
framework (Working / Episodic / Semantic / Procedural). It does NOT modify
|
|
@@ -28,6 +11,11 @@ The full design lives at
|
|
|
28
11
|
`~/dev/research/topics/repo-episodic-memory-framework/repo-episodic-memory-framework.md`
|
|
29
12
|
(see §11–§14).
|
|
30
13
|
|
|
14
|
+
Companion scripts: `scripts/knowledge_review.py` (aggregates all four sections into a
|
|
15
|
+
markdown report), `scripts/detect_decision_rot.py` (rot section),
|
|
16
|
+
`scripts/procedural_governance.py` (stale-procedures section, validate-symbols mode),
|
|
17
|
+
`scripts/consolidate_memory.py` (next-step action when surface items accumulate).
|
|
18
|
+
|
|
31
19
|
## What it surfaces
|
|
32
20
|
|
|
33
21
|
```
|
|
@@ -101,7 +89,7 @@ records the action in `.semantic/_candidates_history.jsonl`.
|
|
|
101
89
|
|
|
102
90
|
## Read-only contract
|
|
103
91
|
|
|
104
|
-
This
|
|
92
|
+
This surface never:
|
|
105
93
|
- promotes items from `_review/` (user does the `mv`)
|
|
106
94
|
- mutates `last_validated` (user edits the file)
|
|
107
95
|
- resolves conflicts (user updates the rows)
|
|
@@ -157,15 +157,31 @@ After the instrumentation lands:
|
|
|
157
157
|
3. If tests still fail silently → instrumentation did not solve the visibility problem; escalate to user
|
|
158
158
|
4. **Always revert** at session end unless the user explicitly approved keep-in-diff via the prompt above. The orchestrator (or caller) verifies no `build-loop:trace/` stash entries remain and no unguarded trace calls landed.
|
|
159
159
|
|
|
160
|
-
## Extended capability —
|
|
160
|
+
## Extended capability — Coding Debugger escalation
|
|
161
161
|
|
|
162
|
-
|
|
162
|
+
When the bundled tier selection / codegen / placement isn't enough — the project requires a tracer backend or placement intelligence that lives only in the standalone Coding Debugger plugin, or you need cross-build log correlation — this skill escalates directly (folded in from the former `logging-tracer-bridge` skill, 2026-07, pool-consolidation Inc 4). This is a secondary, outbound-only hop: the orchestrator owns when-to-fire (Phase 1 Assess observability scan, Phase 5 Iterate `evidence_gap` trigger) and routes to this skill; this skill owns tier selection / codegen / ephemeral policy; the escalation below is optional.
|
|
163
|
+
|
|
164
|
+
**Pre-flight (always run first):**
|
|
163
165
|
|
|
164
166
|
```
|
|
165
|
-
|
|
167
|
+
if (!state.availablePlugins.codingDebugger) {
|
|
168
|
+
return { delegated: false, reason: "standalone Coding Debugger plugin not installed" }
|
|
169
|
+
}
|
|
166
170
|
```
|
|
167
171
|
|
|
168
|
-
|
|
172
|
+
If the standalone plugin is not installed, continue with bundled-only Tier 1/2/3 capability — never hard-fail (graceful degradation).
|
|
173
|
+
|
|
174
|
+
**Delegations available** (pass through the caller-supplied `{ symptom, target_files, tier_hint }`; fold the enriched result back into this skill's own codegen):
|
|
175
|
+
|
|
176
|
+
| Capability needed | Standalone Skill / MCP call |
|
|
177
|
+
|---|---|
|
|
178
|
+
| Extended tracer backends not in bundle | `Skill("coding-debugger:logging-tracer")` with `tier: <upstream-only>` |
|
|
179
|
+
| Cross-build log correlation (correlation IDs across multiple build-loop runs) | standalone-only MCP tools |
|
|
180
|
+
| Advanced placement intelligence (function-call-graph-aware insertion) | standalone-only assessor skills |
|
|
181
|
+
|
|
182
|
+
**Does NOT:** reimplement tier selection / stack detection / ephemeral mechanisms / placement (those stay in this skill); replace the orchestrator's when-to-fire policy; introduce new logging dependencies without explicit user approval; mutate `.build-loop/issues/`; hard-fail when standalone is absent.
|
|
183
|
+
|
|
184
|
+
**State:** optional escalations log to `.build-loop/state.json.observability.escalations[]` — `{ "ts": "ISO", "calledBy": "logging-tracer", "reason": "tier_3_otel_required", "delegated": true|false }`.
|
|
169
185
|
|
|
170
186
|
## Log Analysis Guidance
|
|
171
187
|
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: model-bakeoff
|
|
3
|
+
description: Use to run a controlled multi-model bake-off — have N models (e.g. Opus 4.8, Sonnet 5.0, GPT-5.5) each independently diagnose→plan→execute the SAME bounded change in isolated git worktrees, then deterministically re-score their committed code on a fresh server, merge the best-of (grafting distinct wins from the others), and repeat per change. Triggers on "bake-off", "compare models on this task", "which model is best at", "run the same change across models and score them". NOT for choosing a model tier/segment for one agent — use `model-tiering`.
|
|
4
|
+
user-invocable: false
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Model Bake-off Harness
|
|
8
|
+
|
|
9
|
+
Run a fair, evidence-based competition where several models each solve the *same* change end-to-end, then merge the best result. One orchestrator (this session) coordinates; contestants are single agents (measure the model, not a multi-agent loop). Repeat per change, accumulating merges on one experiment branch.
|
|
10
|
+
|
|
11
|
+
## Roster & dispatch (verified handles)
|
|
12
|
+
- Opus 4.8 → `Agent(model: "opus")`; Sonnet 5.0 → `Agent(model: "sonnet")` (`sonnet` = latest, NOT 4.x — older Sonnets have no clean subagent handle).
|
|
13
|
+
- GPT-5.5 → Codex MCP `mcp__codex__codex` with `model: "gpt-5.5"`, `config: {model_reasoning_effort: "xhigh", sandbox_workspace_write:{network_access:true}}`, `approval-policy: "never"`, `sandbox: "workspace-write"`. (Check `~/.codex/config.toml` for the exact model id; `-codex` suffixes fail on ChatGPT-account Codex.)
|
|
14
|
+
- Independent judge: prefer a NON-contestant model (e.g. Fable). If unavailable, the orchestrator scores subjective dims with over-cited evidence + a stated caveat, and leans on deterministic dims.
|
|
15
|
+
|
|
16
|
+
## Per-change protocol
|
|
17
|
+
1. **Baseline:** branch the experiment off clean `origin/main` (not a dirty/active branch). Confirm no concurrent session collides.
|
|
18
|
+
2. **Scaffold** one worktree per contestant off the experiment branch HEAD: `git worktree add -b bakeoff/<Cn>-<model> <path> <branch>`; `npm ci` (or lockfile-equiv) per worktree; copy `.env.local`.
|
|
19
|
+
3. **Brief** (IDENTICAL for all): give the *symptom* + acceptance criteria + a fair equal entrypoint pointer — WITHHOLD the diagnosis (that's what's scored). Add repo guardrails (see below).
|
|
20
|
+
4. **Dispatch** all contestants in parallel (Agent arms `run_in_background: true`; Codex arm blocks the turn — fine, the others run concurrently).
|
|
21
|
+
5. **Commit stranded Codex work:** Codex's sandbox usually cannot write an external worktree's `.git` (`index.lock: Operation not permitted`). The orchestrator commits it: `git -C <worktree> add -A && git commit`. (RESULT.md is often gitignored → `git add -f`.)
|
|
22
|
+
6. **Score deterministically** (don't trust self-reports — re-run each contestant's committed code):
|
|
23
|
+
- Objective dims computed in code: typecheck, build, test, betterer, + the change-specific success criterion run on a fresh server. Weight these highest.
|
|
24
|
+
- Subjective dims (diagnosis depth, plan, code quality, intent fidelity) by the judge reading diffs + captured outputs.
|
|
25
|
+
7. **Scorecard** (rubric below) → **merge** best base onto the experiment branch, **grafting** distinct wins from the others (each graft: fixes a real gap the winner has, is isolable from the loser's *harmful* parts, verified by re-running). Document graft rationale.
|
|
26
|
+
8. **Re-verify the merged result**, regenerate coverage, commit. Then next change.
|
|
27
|
+
|
|
28
|
+
## Rubric (max 50; tune weights per task)
|
|
29
|
+
success-criteria attainment ×3 (objective) · build/typecheck/test/betterer ×2 (objective) · diagnosis accuracy ×2 · plan quality ×1 · code quality+scope ×1 · intent fidelity ×1. Objective dims dominate; the LLM judge is confined to subjective dims.
|
|
30
|
+
|
|
31
|
+
## Hard-won lessons (do these or the scoring is wrong)
|
|
32
|
+
- **UI-faithful inputs.** Score with inputs the real UI can actually send. (A driver that sent an out-of-range `timeHorizonDays` the UI caps at 90 unfairly zeroed 2 of 3 contestants whose validators — correctly — rejected it.)
|
|
33
|
+
- **Dynamic free-port allocation; NEVER blanket-kill by port.** External processes steal fixed ports; `pkill`-by-port killed *peer contestants'* live dev servers mid-run twice. Allocate a guaranteed-free port per server, check before bind, kill only your own PIDs.
|
|
34
|
+
- **Multi-sample runtime scoring.** LLM output is non-deterministic (temp>0). A single sample misled once (a model's terse run read as failure). Take ≥3 samples for the pass/fail criterion; report pass-rate.
|
|
35
|
+
- **Write-path changes contaminate a shared DB.** For UI/render/summary fixes that self-heal or persist, contestants' writes to a shared DB poison each other's before/after and self-heal the very rows you test. Prefer **function-level tests** (feed the exact bad input through each contestant's exported cleaner — no DB writes). For write-heavy/auth stages, DO NOT share a live DB (per-contestant schema/branch, or serialize); require a UNIQUE test-user id per contestant and id-scoped (not suffix-scoped) cleanup.
|
|
36
|
+
- **Verify against the REAL path, not the self-reported one.** A contestant's "5/5 pass" exercised a code path real traffic doesn't; the real-UI re-run showed 0/3. Re-run the actual user flow.
|
|
37
|
+
- **Betterer/coverage baseline.** Fresh worktrees need `npm run test:coverage` before betterer (coverage-summary.json). On merge, keep the repo-level baseline; `betterer --update` only to *include new tested code* — NEVER bake in a *lowered* baseline caused by skipped/failing tests (that weakens the guardrail for everyone).
|
|
38
|
+
- **No schema migrations against a shared DB.** Forbid `prisma migrate`/`db push`; use existing columns / JSON blobs.
|
|
39
|
+
|
|
40
|
+
## Repo guardrails to put in every brief
|
|
41
|
+
Work ONLY in your worktree; no edits outside it; no deploy/push; no `--no-verify`; no DDL against the shared DB; unique per-model test-user id + id-scoped cleanup; verify by RUNNING (name the exact verification mechanism: curl the endpoint, CDP virtual authenticator for WebAuthn, function-level test for cleaners, screenshots for UI).
|
|
42
|
+
|
|
43
|
+
## Output
|
|
44
|
+
Per stage: a `SCORECARD.md` (rubric table + verdict + merge/graft rationale). At the end: a consolidated `RESULTS.md` (cross-stage scoreboard, per-model performance pattern, where multi-model merge beat any single model, scoring-integrity caveats).
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: model-tiering
|
|
3
|
-
description: Use when choosing a model tier for a subagent, deciding
|
|
3
|
+
description: Use when choosing a model tier or segment for a subagent, deciding a role descriptor (segment + tier) in frontmatter, or escalating mid-flow. Covers the two-axis taxonomy (work-role segment × 7-rung capability ladder) — Opus/Sonnet/Haiku are Anthropic-default mappings; selection is provider-portable and data-driven. NOT for benchmarking models head-to-head on a task — use `model-bakeoff`.
|
|
4
4
|
user-invocable: false
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -13,13 +13,22 @@ Governs model selection across all build-loop phases. Build-loop is **multi-mode
|
|
|
13
13
|
Use `references/agent-role-taxonomy.md` for responsibility boundaries. This
|
|
14
14
|
skill answers "which tier should run the role?", not "who owns the work?".
|
|
15
15
|
|
|
16
|
-
##
|
|
16
|
+
## Two-axis taxonomy (the source of truth)
|
|
17
|
+
|
|
18
|
+
Selection runs on **two orthogonal axes**, encoded as data in `references/model-taxonomy.json` (loader: `scripts/model_taxonomy.py`):
|
|
19
|
+
|
|
20
|
+
- **SEGMENT** — the work role / primary output: Generative Reasoning, Agentic Execution, Representation/Retrieval, Realtime Interaction, Perception/Input Understanding, Generative Media, Governance/Evaluation. Segment is the *primary product role* — a reasoning model that accepts image/audio input is Generative Reasoning with a `multimodal-input` tag, not Perception.
|
|
21
|
+
- **CAPABILITY-TIER** — a 7-rung ladder: T0 experimental/restricted frontier · T1 ultra-frontier · T2 frontier · T3 balanced workhorse · T4 efficient near-frontier · T5 utility/nano/edge · T-S specialist infrastructure (off the ladder).
|
|
22
|
+
|
|
23
|
+
Agents declare a `(segment, tier)` ROLE; the resolver (`scripts/model_resolver.py resolve_role`) walks the per-cell ordered preferred list and returns the highest-ranked AVAILABLE + host-reachable model (ties → release recency). New models are classified once (`scripts/classify_model_tier.py`, host-LLM, both axes) — no agent edits. **The 4-tier table below is the legacy alias view** — `frontier/thinking/code/pattern` fold to `T1/T2/T3/T4` (Generative Reasoning segment) and remain accepted everywhere for back-compat. Dormant segments (Realtime/Perception/Media) are data + reference only — no resolver wiring yet.
|
|
24
|
+
|
|
25
|
+
## Tier abstraction (legacy 4-token alias view — Generative Reasoning T1–T4)
|
|
17
26
|
|
|
18
27
|
| Tier | Anthropic default | Role | Equivalents (advisory — verify benchmarks before swapping) |
|
|
19
28
|
|---|---|---|---|
|
|
20
29
|
| **Frontier** | Fable 5 | **Phase 2 Plan synthesis (frame goal, draft spec/ADRs, F-criteria, MECE partition) via the Advisor dispatch ladder when stakes-gated** — `advisor` agent / peer host / already-Fable session; honestly-labeled inline-Opus fallback otherwise (`references/advisor-dispatch-ladder.md`). (Advisor v1 = Phase 2 only; Phase 1 Assess synthesis runs inline as today until v2.) AND verification judgment (plan-critic, scope-auditor, independent-auditor, fix-critique, fact-checker, security-reviewer, overfitting-reviewer, promotion-reviewer) | GPT-5.5 Thinking (or whichever tier scores above the prior Thinking-tier ceiling), future Claude tier above Opus; any model that benchmarks above the Thinking-tier contract on SWE-bench Verified AND ARC-AGI / GPQA Diamond |
|
|
21
30
|
| **Thinking** | Opus 4.8 | Coordination — build-orchestrator, assessment-orchestrator — and the escalation target for execution (ambiguous spec, 2 consecutive failures, cross-file surprise) and audit/learnings synthesis when Frontier is unavailable | GPT-5 Thinking, Gemini 2.5 Pro; any model >= Opus 4.6 on SWE-bench Verified + Frontier-class on ARC-AGI / MMLU-Pro |
|
|
22
|
-
| **Code** | Sonnet
|
|
31
|
+
| **Code** | Sonnet 5 | Application — apply rule to bounded input, scoped implementation, mechanical refactor, bounded domain assessment | GPT-5 Codex, qwen2.5-coder-32B (local); any model with SWE-bench Verified within ~5pt of the Code-tier default (last published Anthropic Sonnet figure: 4.6 ~79.6%; Sonnet 5 reaches prior Opus-tier coding/agentic quality per claude-api T1) |
|
|
23
32
|
| **Pattern** | Haiku 4.5 | Recognition — regex/syntactic match, classification into known buckets, log scan, deterministic checklist | Haiku 4.6, GPT-5 Mini, llama3.2-3b (local); any small/fast model that handles structured pattern matching |
|
|
24
33
|
|
|
25
34
|
**Rule of substitution:** tier A's swap target must score within tolerance of the default on the benchmark relevant to its role. For Code tier that's SWE-bench Verified ≥75% AND tool-use accuracy ≥85%; for Thinking tier that's SWE-bench ≥78% AND ARC-AGI / GPQA Diamond competitive; for Frontier tier that's clearing the Thinking-tier contract AND scoring above the prior-generation Thinking-tier ceiling on at least one of SWE-bench Verified / ARC-AGI / GPQA Diamond; for Pattern tier no benchmark — just "fast and cheap, doesn't hallucinate on bounded structured tasks."
|
|
@@ -30,12 +39,31 @@ skill answers "which tier should run the role?", not "who owns the work?".
|
|
|
30
39
|
|
|
31
40
|
Build-loop's agent frontmatter uses Anthropic model aliases (`fable`, `opus`, `sonnet`, `haiku`) because Claude Code is the primary host. To run on a different provider:
|
|
32
41
|
|
|
33
|
-
1. **
|
|
42
|
+
1. **Edit the INDEX, not each agent.** `model:` frontmatter is index-DERIVED (generated by `scripts/sync_agent_model_defaults.py`), so do not hand-edit it. To swap providers, reorder the preferred list / change the default for the `(segment, tier)` cell in `references/model-taxonomy.json` (or classify a new model once via `scripts/classify_model_tier.py`), then run `python3 scripts/sync_agent_model_defaults.py --apply` to regenerate every agent's `model:`. The role's `(segment, tier)` is the durable key; the tier determines the substitution target.
|
|
34
43
|
2. **Runtime override:** `.build-loop/config.json.modelOverrides` accepts `{ frontier: "<id>", thinking: "<id>", code: "<id>", pattern: "<id>" }`. The orchestrator resolves this through `scripts/model_overrides.py` before dispatching subagents (see `references/model-tier-mapping.md` for full schema). Configs without `frontier` resolve frontier → `fable` by default.
|
|
35
44
|
3. **Per-dispatch override:** any orchestrator dispatch may pass `model: <id>` in the subagent prompt to force that call.
|
|
36
45
|
|
|
37
46
|
The role-and-task table below uses tier names. The Anthropic-default mapping in the right column is illustrative; substitute your equivalents at swap time.
|
|
38
47
|
|
|
48
|
+
## Chat-triggered index maintenance (host-LLM-driven)
|
|
49
|
+
|
|
50
|
+
The model index (`references/model-taxonomy.json`) is **user-editable and chat-maintainable**. When the user expresses model intent in conversation, recognize it and act on the index directly — this is host-LLM-driven per the repo's "host coding agent is the LLM" rule: you recognize the intent and run deterministic scripts; there is NO vendor API call and NO hard hook. (A `UserPromptSubmit` hook that pre-detects these phrasings is an OPTIONAL future hardening, not required — the LLM recognizing intent is the mechanism.)
|
|
51
|
+
|
|
52
|
+
**Trigger phrasings (illustrative, not exhaustive):** "check the model(s)", "is there a newer model", "what's the current frontier model", "change the `<tier>`/`<segment>` model", "use `<model>` for `<role>`", "what model is `<agent/tier>` using", "swap `<model>` in", "reorder the preferred list".
|
|
53
|
+
|
|
54
|
+
**On a CHECK / NEWER intent** ("check the models", "is there a newer model", "what is X using"):
|
|
55
|
+
1. Read the index — `python3 scripts/model_taxonomy.py --segment <seg> --tier <tier>` for one cell, or `--json` for the summary.
|
|
56
|
+
2. Report the CURRENTLY RECOMMENDED model vs what is AVAILABLE: run `python3 scripts/resolve_agent_model.py <agent> --json` (or `model_resolver.py --segment <s> --tier <t> --json`) and read back `model` + `resolution_path`.
|
|
57
|
+
3. If the user names a model the index does not know, OFFER to classify it via the existing host-LLM flow: `python3 scripts/classify_model_tier.py lookup <id>` returns a WebSearch query + parse rubric; you run the search, decide segment + tier, then `record <id> --tier <tier> --segment <seg> --provider <vendor> [--provenance verified]`. No vendor API call — you (the host LLM) do the interpretation.
|
|
58
|
+
|
|
59
|
+
**On a CHANGE / USE intent** ("use gpt-5.5 for frontier", "change the code model to X", "make sonnet first"):
|
|
60
|
+
1. Edit the index `references/model-taxonomy.json` with the smallest change: reorder the `preferred[<segment>][<tier>]` list (capability-rank order — the first available wins), change a cell's default, or add an already-classified model id. A documented jsonpatch-style single-field edit is enough — no helper script needed (KISS). If the model is not yet classified, classify it first (step above).
|
|
61
|
+
2. Regenerate the derived defaults: `python3 scripts/sync_agent_model_defaults.py --apply`. This rewrites every affected agent's `model:` from the new index state (only harness-valid tokens are written; a cross-provider recommendation keeps the existing token and is reported).
|
|
62
|
+
3. Confirm: `python3 scripts/sync_agent_model_defaults.py --check` returns 0 drift; report the changed agents back.
|
|
63
|
+
|
|
64
|
+
Dispatch always resolves the role LIVE through `resolve_agent_model.py`, so an index edit takes effect on the next dispatch even before a sync — `sync` only keeps the on-disk `model:` fallback honest.
|
|
65
|
+
|
|
66
|
+
|
|
39
67
|
## When to use this skill
|
|
40
68
|
|
|
41
69
|
- Choosing `model:` field in an agent frontmatter
|
|
@@ -48,10 +76,12 @@ The role-and-task table below uses tier names. The Anthropic-default mapping in
|
|
|
48
76
|
|
|
49
77
|
| Claim | Source | Certainty |
|
|
50
78
|
|-------|--------|-----------|
|
|
51
|
-
| Sonnet
|
|
52
|
-
|
|
|
53
|
-
| Sonnet 4.6
|
|
54
|
-
|
|
|
79
|
+
| Sonnet 5 reaches prior Opus-tier quality on coding/agentic work (Code-tier default since 2026-06) | claude-api skill cache (T1 — Anthropic) | ✅ T1 qualitative; no single SWE-bench figure published yet |
|
|
80
|
+
| Sonnet 5 uses a new tokenizer — ~30% more tokens for the same text vs Sonnet 4.6 | claude-api skill cache (T1 — Anthropic) | ✅ T1 — re-baseline token budgets, do not reuse 4.6 counts |
|
|
81
|
+
| (prior-gen baseline) Sonnet 4.6: 79.6% SWE-bench Verified | Anthropic announcement + SWE-bench leaderboard | ⚠️ T2, single-source; historical calibration |
|
|
82
|
+
| (prior-gen baseline) Opus 4.6: 80.8% SWE-bench Verified (1.2pt gap — smallest in Claude history) | Same | ⚠️ T2, single-source; historical |
|
|
83
|
+
| (prior-gen baseline) Sonnet 4.6 uses 70% fewer tokens than 4.5 on complex file ops with +38% accuracy | Anthropic Sonnet 4.6 announcement | ⚠️ T2, single-source; superseded by Sonnet 5's new tokenizer |
|
|
84
|
+
| Pricing: Sonnet 5 $3/$15 per MTok ($2/$10 introductory through 2026-08-31) | claude-api skill cache 2026-06-24 (T1 — Anthropic) | ⚠️ verify before billing |
|
|
55
85
|
| Pricing: Opus 4.8 $5/$25 per MTok input/output | Anthropic pricing page | ⚠️ verify before billing |
|
|
56
86
|
| Pricing: Fable 5 $10/$50 per MTok input/output (1M context, capability tier above Opus 4.8) | claude-api skill cache 2026-05-26 (T1 — Anthropic) | ✅ T1 source, advisory until re-confirmed at next billing audit |
|
|
57
87
|
|
|
@@ -142,7 +172,7 @@ If the ambiguity surfaces a **planning** problem (the original plan no longer fi
|
|
|
142
172
|
- **Self-refine with external verification** (tests, lint, type-check). External oracle is non-negotiable — without it, self-refine is circular.
|
|
143
173
|
- **Adversarial critic loop** (writer agent + read-only reviewer agent). Separation is what makes it work. Same model reviewing its own output doesn't catch errors.
|
|
144
174
|
- **Best-of-N sampling with self-certainty voting** on HARD chunks only (flagged by plan or first-pass failure). N=3. Cost = ~3x Sonnet, still under 1.5x single-pass Opus.
|
|
145
|
-
- **Test-time compute** (effort=
|
|
175
|
+
- **Test-time compute** (effort=high default, effort=xhigh on retry — Sonnet 5 is the first Sonnet with `xhigh`; coding/agentic work wants high/xhigh per claude-api T1). Easier problems benefit from revisions; harder problems need parallel sampling — not just more thinking on one path.
|
|
146
176
|
- **Plan-then-execute split** (Opus plans once, Sonnet executes many). Established pattern. Amortizes Opus cost across N subagent calls.
|
|
147
177
|
|
|
148
178
|
## Techniques to avoid
|
|
@@ -156,15 +186,19 @@ If the ambiguity surfaces a **planning** problem (the original plan no longer fi
|
|
|
156
186
|
|
|
157
187
|
> ⚠️ **Advisory only.** The numbers below are directional heuristics based on single-source token-profile estimates and public pricing as of the skill's last update. They are **not** verified against real usage telemetry and should not be used as hard routing logic. Treat them as "this tier costs roughly this much more than that tier," not as commitments. Pricing, token profiles, and model output lengths all drift over time. Before using these ratios in any cost-minimization decision, pull actual usage data from the last 30 days of builds and re-derive the numbers for your workload.
|
|
158
188
|
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
|
162
|
-
|
|
163
|
-
| Sonnet 4.
|
|
164
|
-
| Sonnet
|
|
165
|
-
|
|
|
189
|
+
**Price ratios only** (T1, claude-api — per-MTok, in/out): Fable 5 $10/$50 · Opus 4.8 $5/$25 · Sonnet 5 $3/$15 ($2/$10 intro through 2026-08-31) · Haiku 4.5 $1/$5.
|
|
190
|
+
|
|
191
|
+
| Pair (same task) | Price ratio | Basis |
|
|
192
|
+
|---|---|---|
|
|
193
|
+
| Sonnet 5 vs Opus 4.8 | **~0.6x** | $3/$15 ÷ $5/$25 = 0.6 in & out. Sonnet 5 and Opus 4.8 share the **same** new tokenizer (T1), so token counts are comparable — the ratio is price-driven, not token-driven |
|
|
194
|
+
| Sonnet 5 vs Fable 5 | **~0.3x** | $3/$15 ÷ $10/$50 |
|
|
195
|
+
| Haiku 4.5 vs Sonnet 5 | **~0.33x** | $1/$5 ÷ $3/$15 |
|
|
196
|
+
|
|
197
|
+
> ⚠️ **These are price ratios, not measured task costs.** Real per-task cost also depends on token volume and effort — running Sonnet 5 at `high`/`xhigh` (the new default + escalation rung) raises its output-token count, narrowing the gap to Opus. A *measured* multiplier cannot yet be stated: the cost ledger (`~/.bookmark/cost-ledger.jsonl`) is currently dominated by local-model rows with no Anthropic build telemetry. **Re-derivation trigger:** once the ledger carries real Sonnet-5 + Opus build rows, pull the last 30 days and re-derive measured task-cost multipliers per workload.
|
|
198
|
+
>
|
|
199
|
+
> The retired "~0.3x single-pass Sonnet" figure was a Sonnet-**4.6** token-profile artifact ("70% fewer tokens than 4.5") and does **not** carry to Sonnet 5 — whose new tokenizer emits ~30% *more* tokens per unit text than 4.6. That ~30% delta matters only when comparing Sonnet 5 to the old 4.6 numbers, **not** to Opus 4.8 (shared tokenizer).
|
|
166
200
|
|
|
167
|
-
❓ Best-of-N + critic vs single-pass Opus on SWE-bench has not been directly benchmarked.
|
|
201
|
+
❓ Best-of-N + critic vs single-pass Opus on SWE-bench has not been directly benchmarked on Sonnet 5.
|
|
168
202
|
|
|
169
203
|
**How to convert these into routing decisions**: don't. Use the numbers to sanity-check a tier choice after the fact ("was this worth the 5x?"), not to justify forcing a model swap. When real telemetry disagrees with this table, trust telemetry and file an issue to update the table.
|
|
170
204
|
|
|
@@ -172,7 +206,7 @@ If the ambiguity surfaces a **planning** problem (the original plan no longer fi
|
|
|
172
206
|
|
|
173
207
|
**Fable plans (when stakes-gated) and verifies. Opus coordinates. Sonnet executes. Haiku recognizes.**
|
|
174
208
|
|
|
175
|
-
Phase 2 Plan synthesis reaches **Fable** through the **Advisor dispatch ladder** when stakes-gating trips (`synthesisDensity > 5`, `riskSurfaceChange`, `stakes >= medium`, or `dispatch_tier: frontier`): the orchestrator dispatches the `advisor` agent (Rung 1), routes to a peer host (Rung 2), or — if its own session is already Fable — synthesizes inline at Frontier (Rung 0). When no trigger fires or no dispatch path is reachable, the orchestrator synthesizes the plan **inline on its own model (Opus)** and labels it honestly (Rung 3 = today's behavior; the floor equals current state). So "Fable plans" is the *guarantee on high-stakes plans*, with an honestly-labeled inline fallback otherwise — not unconditional. Full protocol: `references/advisor-dispatch-ladder.md`. The Advisor frames the goal, drafts the spec/ADRs, sets F-criteria, and MECE-partitions the work. The orchestrator (**Opus**, `build-orchestrator`, `assessment-orchestrator`) coordinates: it routes dispatches, runs deterministic gates, manages parallel fan-out, walks the Advisor ladder, and handles the escalation ladder. Phase 3 implementer subagents run on **Sonnet** at effort=
|
|
209
|
+
Phase 2 Plan synthesis reaches **Fable** through the **Advisor dispatch ladder** when stakes-gating trips (`synthesisDensity > 5`, `riskSurfaceChange`, `stakes >= medium`, or `dispatch_tier: frontier`): the orchestrator dispatches the `advisor` agent (Rung 1), routes to a peer host (Rung 2), or — if its own session is already Fable — synthesizes inline at Frontier (Rung 0). When no trigger fires or no dispatch path is reachable, the orchestrator synthesizes the plan **inline on its own model (Opus)** and labels it honestly (Rung 3 = today's behavior; the floor equals current state). So "Fable plans" is the *guarantee on high-stakes plans*, with an honestly-labeled inline fallback otherwise — not unconditional. Full protocol: `references/advisor-dispatch-ladder.md`. The Advisor frames the goal, drafts the spec/ADRs, sets F-criteria, and MECE-partitions the work. The orchestrator (**Opus**, `build-orchestrator`, `assessment-orchestrator`) coordinates: it routes dispatches, runs deterministic gates, manages parallel fan-out, walks the Advisor ladder, and handles the escalation ladder. Phase 3 implementer subagents run on **Sonnet** at effort=high (default workhorse; xhigh on hard/code-heavy chunks) → external verification gate (tests/lint/types) → adversarial **Fable** verification surface (`plan-critic`, `scope-auditor`, `independent-auditor`, `fix-critique`, `fact-checker`, `security-reviewer`, `overfitting-reviewer`, `promotion-reviewer`). On a first execution-problem chunk failure, retry the same Sonnet 5 implementer at **effort=xhigh** (intermediate rung — near-Opus coding at lower cost) before paying for Opus; a strong-checkpoint finding or a 2nd consecutive failure after the xhigh retry escalates to **Opus** for judgment; if the failure traces back to a planning miss, route back to Fable to re-plan. See `agents/build-orchestrator.md §Escalation Triggers`. The **tier mapping** is the policy; the cost numbers above are advisory context, not the basis for overrides.
|
|
176
210
|
|
|
177
211
|
Haiku is only used for Phase 7B mock scanning and recurring-pattern detection across `runs[]`. Never for reasoning tasks.
|
|
178
212
|
|
|
@@ -184,11 +218,12 @@ Not every agent should hard-pin its model. Use this rule:
|
|
|
184
218
|
- **Inherit** (`model: inherit`) when user intent should flow through. The user's main-session choice is itself a cost/speed preference; respect it. Pair with a "recommended: X" note in this skill rather than forcing via frontmatter. Example: `root-cause-investigator` — recommended Opus on causal-tree work, but inherit honors whatever tier the user picked upstream.
|
|
185
219
|
- **Override mechanism**: users can override any pin by passing `model:` when spawning the agent or by editing the frontmatter. Pins are defaults, not locks. The deliberate exceptions documented above (`alignment-checker`, `synthesis-critic` on Sonnet despite being verification-shaped) are exactly this kind of cost-vs-judgment pin and can be lifted if telemetry says so.
|
|
186
220
|
|
|
187
|
-
Forward-compat note: pinned family aliases (`fable`, `sonnet`, `opus`) auto-track latest versions in their tier (e.g., Sonnet 4.6 →
|
|
221
|
+
Forward-compat note: pinned family aliases (`fable`, `sonnet`, `opus`) auto-track latest versions in their tier (e.g., Sonnet 4.6 → 5, Opus 4.7 → 4.8, Fable 5 → 6). `inherit` additionally picks up brand-new tiers (e.g., a future Flash-class model) without frontmatter edits.
|
|
188
222
|
|
|
189
223
|
## Limitations of this guidance
|
|
190
224
|
|
|
191
|
-
- ⚠️ Sonnet 4.6 token-efficiency claim is single-source (Anthropic announcement). Treat as directionally correct, not proven.
|
|
225
|
+
- ⚠️ Sonnet 4.6 token-efficiency claim is single-source (Anthropic announcement). Treat as directionally correct, not proven; superseded for the active default by Sonnet 5's new tokenizer (~30% more tokens/text, claude-api T1).
|
|
226
|
+
- ❓ No single published SWE-bench Verified figure for Sonnet 5 yet; the Code-tier capability claim rests on the T1 qualitative "reaches prior Opus-tier coding/agentic quality." Re-confirm with a benchmark when available.
|
|
192
227
|
- ❓ Best-of-N + critic hasn't been tested against single-pass Opus on SWE-bench specifically.
|
|
193
228
|
- ⚠️ Escalation triggers are heuristics, not proven thresholds. Revise after observing 5+ real builds and logging outcomes to `.build-loop/memory/`.
|
|
194
229
|
|
package/skills/optimize/SKILL.md
CHANGED
|
@@ -40,7 +40,7 @@ Highest-leverage phase. Wrong metric = Goodhart's Law. Wrong factors = wasted ru
|
|
|
40
40
|
|---|---|---|
|
|
41
41
|
| **A. Power-user explicit** | User supplied factors via CLI flag, `.build-loop/optimize/factors.json`, or inline ("optimize batch_size, retries, workers for throughput") | Skip suggestion; use the user's factors directly |
|
|
42
42
|
| **B. Vague optimization** *(default)* | "run optimization", "make my app faster", "improve performance", "speed up", "reduce <metric>" without naming factors | Run factor-identification scan; propose candidates; **AskUserQuestion to confirm before running** |
|
|
43
|
-
| **C. Single-variable explicit** | "simplify this file", "reduce build time", scoped `/build-loop:
|
|
43
|
+
| **C. Single-variable explicit** | "simplify this file", "reduce build time", a scoped optimize request via `/build-loop:run <known-target>` | Skip DOE; run autoresearch (existing behavior, Phase 2 LOOP unchanged) |
|
|
44
44
|
|
|
45
45
|
### Step 1.2 — Branch A or B: factor identification
|
|
46
46
|
|
|
@@ -186,7 +186,7 @@ Dispatch the `optimize-runner` agent. It executes:
|
|
|
186
186
|
|
|
187
187
|
Phase 4.7 (AUTO-OPTIMIZE): after Phase 4 Execute completes and commits, check for optimization targets. Run sequentially (not parallel with Phase 4).
|
|
188
188
|
|
|
189
|
-
|
|
189
|
+
Reached via `/build-loop:run [target]` + optimize language (no separate command).
|
|
190
190
|
|
|
191
191
|
## State Files
|
|
192
192
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: build-loop:prd-bridge
|
|
3
|
-
description: Use when Phase 1 Assess runs, the user mentions a PRD, or asks to "ground the build in product strategy". Surfaces always-true principles and Navigation Map from the project PRD so subsequent phases reason from strategic frame; recommends prd-builder if none exists.
|
|
3
|
+
description: Use when Phase 1 Assess runs, the user mentions a PRD, or asks to "ground the build in product strategy". Surfaces always-true principles and Navigation Map from the project PRD so subsequent phases reason from strategic frame; recommends prd-builder if none exists. NOT for authoring a PRD from scratch (use `prd-builder`), API registration (use `api-registry-bridge`), or UI verification (use `ibr-bridge`).
|
|
4
4
|
version: 0.1.0
|
|
5
5
|
user-invocable: false
|
|
6
6
|
---
|