@tyroneross/build-loop 0.35.0 → 0.36.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/marketplace.json +1 -1
- package/.claude-plugin/marketplace.json +3 -3
- package/.claude-plugin/plugin.json +2 -2
- package/.codex-plugin/plugin.json +1 -1
- package/AGENTS.md +67 -14
- package/README.md +208 -299
- package/agents/advisor.md +2 -0
- package/agents/alignment-checker.md +2 -0
- package/agents/api-assessor.md +3 -1
- package/agents/architecture-scout.md +2 -0
- package/agents/assessment-orchestrator.md +2 -0
- package/agents/build-orchestrator.md +13 -8
- package/agents/database-assessor.md +107 -3
- package/agents/design-contract-specialist.md +2 -0
- package/agents/fact-checker.md +2 -0
- package/agents/fix-critique.md +26 -5
- package/agents/frontend-assessor.md +3 -1
- package/agents/implementer.md +2 -0
- package/agents/independent-auditor.md +2 -0
- package/agents/mock-scanner.md +2 -0
- package/agents/optimize-runner.md +2 -0
- package/agents/overfitting-reviewer.md +2 -0
- package/agents/performance-assessor.md +3 -1
- package/agents/plan-critic.md +2 -0
- package/agents/promotion-reviewer.md +2 -0
- package/agents/recurring-pattern-detector.md +4 -1
- package/agents/retrospective-synthesizer.md +14 -0
- package/agents/root-cause-investigator.md +39 -2
- package/agents/scope-auditor.md +2 -0
- package/agents/security-reviewer.md +3 -0
- package/agents/self-improvement-architect.md +2 -0
- package/agents/synthesis-critic.md +2 -0
- package/agents/transcript-pattern-miner.md +2 -0
- package/agents/ui-validator.md +2 -0
- package/assets/build-loop-plugin-icon.png +0 -0
- package/bin/build-loop-install.js +234 -0
- package/commands/run.md +1 -1
- package/hooks/git/pre-commit +40 -0
- package/hooks/git/pre-push +35 -10
- package/hooks/hooks.json +42 -5
- package/hooks/pre-edit-architecture.sh +2 -2
- package/hooks/route-guard.sh +15 -0
- package/hooks/route_guard.py +221 -0
- package/hooks/session-start-extensions.sh +10 -0
- package/hooks/session-start-git-hooks.sh +6 -1
- package/hooks/session-start-rally-point.sh +23 -1
- package/hooks/test_session_start_rally_version.sh +123 -0
- package/package.json +7 -2
- package/plugin-artifacts/codex/.codex-plugin/plugin.json +1 -1
- package/plugin-artifacts/codex/AGENTS.md +67 -14
- package/plugin-artifacts/codex/README.md +208 -299
- package/plugin-artifacts/codex/assets/build-loop-plugin-icon.png +0 -0
- package/plugin-artifacts/codex/references/backlog-system.md +285 -0
- package/plugin-artifacts/codex/references/capability-routing.md +49 -0
- package/plugin-artifacts/codex/references/coordination-rules.md +271 -2
- package/plugin-artifacts/codex/references/memory-systems.md +25 -2
- package/plugin-artifacts/codex/references/memory.md +14 -0
- package/plugin-artifacts/codex/references/model-tier-mapping.md +45 -6
- package/plugin-artifacts/codex/references/phase-1-assess.md +19 -1
- package/plugin-artifacts/codex/references/phase-2-plan.md +9 -1
- package/plugin-artifacts/codex/references/phase-3-execute.md +1 -1
- package/plugin-artifacts/codex/references/phase-4-review.md +20 -1
- package/plugin-artifacts/codex/references/phase-5-iterate.md +4 -3
- package/plugin-artifacts/codex/references/phase-6-learn.md +2 -0
- package/plugin-artifacts/codex/references/verify-dispatch.md +85 -0
- package/plugin-artifacts/codex/skills/build-loop/SKILL.md +19 -10
- package/plugin-artifacts/codex/skills/build-loop/fallbacks.md +1 -1
- package/plugin-artifacts/codex/skills/build-loop/references/capability-routing.md +49 -0
- package/plugin-artifacts/codex/skills/build-loop/references/experiment-results-template.md +101 -0
- package/plugin-artifacts/codex/skills/build-loop/references/memory.md +14 -0
- package/plugin-artifacts/codex/skills/build-loop/references/output-style.md +98 -5
- package/plugin-artifacts/codex/skills/build-loop/references/phase-1-assess.md +19 -1
- package/plugin-artifacts/codex/skills/build-loop/references/phase-2-plan.md +9 -1
- package/plugin-artifacts/codex/skills/build-loop/references/phase-3-execute.md +1 -1
- package/plugin-artifacts/codex/skills/build-loop/references/phase-4-review.md +20 -1
- package/plugin-artifacts/codex/skills/build-loop/references/phase-5-iterate.md +4 -3
- package/plugin-artifacts/codex/skills/build-loop/references/phase-6-learn.md +2 -0
- package/plugin-artifacts/codex/skills/build-loop/references/verify-dispatch.md +85 -0
- package/scripts/build_codex_plugin_artifact.py +7 -0
- package/scripts/embed_backend.py +17 -13
- package/scripts/lessons_index/query.py +1 -1
- package/scripts/memory_graph/__init__.py +2 -2
- package/scripts/sync_navgator_lessons.py +4 -4
- package/skills/api-registry-bridge/SKILL.md +1 -1
- package/skills/auto-decision-capture/SKILL.md +4 -3
- package/skills/auto-finding-capture/SKILL.md +119 -0
- package/skills/build-loop/SKILL.md +19 -10
- package/skills/build-loop/fallbacks.md +1 -1
- package/skills/build-loop/references/capability-routing.md +49 -0
- package/skills/build-loop/references/experiment-results-template.md +101 -0
- package/skills/build-loop/references/memory.md +14 -0
- package/skills/build-loop/references/output-style.md +98 -5
- package/skills/build-loop/references/phase-1-assess.md +19 -1
- package/skills/build-loop/references/phase-2-plan.md +9 -1
- package/skills/build-loop/references/phase-3-execute.md +1 -1
- package/skills/build-loop/references/phase-4-review.md +20 -1
- package/skills/build-loop/references/phase-5-iterate.md +4 -3
- package/skills/build-loop/references/phase-6-learn.md +2 -0
- package/skills/build-loop/references/verify-dispatch.md +85 -0
- package/skills/debug-loop/SKILL.md +4 -2
- package/skills/debugging-memory/SKILL.md +24 -12
- package/skills/{debugging/assess/SKILL.md → debugging-memory/references/assess.md} +6 -13
- package/skills/{debugging/memory/SKILL.md → debugging-memory/references/search.md} +10 -17
- package/skills/{debugging/store/SKILL.md → debugging-memory/references/store.md} +5 -13
- package/skills/debugging-memory/references/subagent-integration.md +3 -3
- package/skills/defenseclaw-bridge/SKILL.md +1 -1
- package/skills/{loop-builder → focused-loop-builder}/SKILL.md +6 -6
- package/skills/ibr-bridge/SKILL.md +1 -1
- package/skills/knowledge/SKILL.md +13 -2
- package/skills/{knowledge-review/SKILL.md → knowledge/references/review-mode.md} +8 -20
- package/skills/logging-tracer/SKILL.md +20 -4
- package/skills/model-bakeoff/SKILL.md +44 -0
- package/skills/model-tiering/SKILL.md +55 -20
- package/skills/optimize/SKILL.md +2 -2
- package/skills/prd-bridge/SKILL.md +1 -1
- package/skills/recursive-retrospective/SKILL.md +46 -0
- package/skills/recursive-retrospective/references/01-retrospective.md +136 -0
- package/skills/recursive-retrospective/references/02-learning-capture.md +72 -0
- package/skills/recursive-retrospective/references/03-judge.md +45 -0
- package/skills/research/SKILL.md +1 -1
- package/skills/root-cause-analysis/SKILL.md +42 -0
- package/skills/security-methodology/SKILL.md +10 -1
- package/skills/security-scan/SKILL.md +55 -0
- package/skills/spec-writing/SKILL.md +42 -1
- package/skills/sync-skills/SKILL.md +8 -7
- package/commands/assess.md +0 -41
- package/commands/compose-handoff.md +0 -113
- package/commands/debug.md +0 -20
- package/commands/debugger-detail.md +0 -34
- package/commands/debugger-scan.md +0 -15
- package/commands/debugger-status.md +0 -20
- package/commands/debugger.md +0 -32
- package/commands/optimize-run.md +0 -31
- package/commands/promote-experiment.md +0 -129
- package/commands/rally-point.md +0 -301
- package/commands/research-run.md +0 -22
- package/commands/review-knowledge.md +0 -16
- package/commands/self-review.md +0 -51
- package/commands/setup-memory.md +0 -32
- package/commands/start-prd.md +0 -57
- package/commands/test.md +0 -34
- package/commands/verify-plan.md +0 -39
- package/skills/debugging/debug-loop/SKILL.md +0 -231
- package/skills/logging-tracer-bridge/SKILL.md +0 -70
- /package/skills/{loop-builder → focused-loop-builder}/presets/active-project-evidence.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/presets/generic-artifact-loop.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/presets/presentation-audit.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/presets/research-synthesis.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/presets/source-ingestion-raw-data-audit.yaml +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/references/spec-format.md +0 -0
- /package/skills/{loop-builder → focused-loop-builder}/scripts/loop_builder.py +0 -0
|
@@ -24,6 +24,7 @@ DEFAULT_TARGET = Path("plugin-artifacts/codex")
|
|
|
24
24
|
IGNORED_NAMES = {".DS_Store", "__pycache__", ".pytest_cache"}
|
|
25
25
|
IGNORED_SUFFIXES = {".pyc"}
|
|
26
26
|
TOP_LEVEL_FILES = ("AGENTS.md", "README.md", "LICENSE")
|
|
27
|
+
ASSET_FILES = (Path("assets") / "build-loop-plugin-icon.png",)
|
|
27
28
|
|
|
28
29
|
# Bundle markdown points readers at ``references/<file>.md`` (root-relative).
|
|
29
30
|
# The Claude source resolves that logical namespace across the repo's top-level
|
|
@@ -226,6 +227,10 @@ def validate_artifact(target: Path) -> None:
|
|
|
226
227
|
if rel_files != ["skills/build-loop/SKILL.md"]:
|
|
227
228
|
raise ArtifactError(f"artifact must expose exactly one SKILL.md, got {rel_files}")
|
|
228
229
|
|
|
230
|
+
for rel_path in ASSET_FILES:
|
|
231
|
+
if not (target / rel_path).is_file():
|
|
232
|
+
raise ArtifactError(f"artifact missing asset: {rel_path}")
|
|
233
|
+
|
|
229
234
|
check_reference_pointers(target)
|
|
230
235
|
|
|
231
236
|
|
|
@@ -242,6 +247,8 @@ def build_artifact(source: Path, target: Path) -> None:
|
|
|
242
247
|
write_codex_manifest(source, tmp / ".codex-plugin" / "plugin.json")
|
|
243
248
|
for name in TOP_LEVEL_FILES:
|
|
244
249
|
copy_file(source / name, tmp / name)
|
|
250
|
+
for rel_path in ASSET_FILES:
|
|
251
|
+
copy_file(source / rel_path, tmp / rel_path)
|
|
245
252
|
copy_tree(source / "skills" / "build-loop", tmp / "skills" / "build-loop")
|
|
246
253
|
copy_file(source / "docs" / "agent-surface-policy.md", tmp / "docs" / "agent-surface-policy.md")
|
|
247
254
|
write_notice(tmp / "BUILD-ARTIFACT.md")
|
package/scripts/embed_backend.py
CHANGED
|
@@ -17,12 +17,13 @@ vectors are NOT comparable, so callers writing rows must record
|
|
|
17
17
|
written by a different model. See research entry
|
|
18
18
|
`build-loop-search-architecture` for rationale.
|
|
19
19
|
|
|
20
|
-
The active backend is chosen on first call
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
20
|
+
The active backend is chosen on first call. When $EMBED_BACKEND is unset,
|
|
21
|
+
a warm local daemon is preferred; an explicit $EMBED_BACKEND
|
|
22
|
+
({"mlx","ollama"}) bypasses the daemon and selects the requested in-process
|
|
23
|
+
backend. If MLX init fails (import error, model load error, first-call
|
|
24
|
+
failure), the module logs a warning to stderr and falls through to Ollama
|
|
25
|
+
for the rest of the process. Once fallen through, MLX is not retried —
|
|
26
|
+
keeps stop-hook latency predictable.
|
|
26
27
|
|
|
27
28
|
Public API:
|
|
28
29
|
embed(text) -> list[float] (single text)
|
|
@@ -372,16 +373,19 @@ def _select_backend():
|
|
|
372
373
|
if _BACKEND is not None:
|
|
373
374
|
return _BACKEND
|
|
374
375
|
|
|
375
|
-
|
|
376
|
-
# selection — we don't load MLX/Ollama in-process at all.
|
|
377
|
-
daemon = _probe_daemon()
|
|
378
|
-
if daemon is not None:
|
|
379
|
-
_BACKEND = daemon
|
|
380
|
-
return _BACKEND
|
|
381
|
-
|
|
376
|
+
explicit_backend = "EMBED_BACKEND" in os.environ
|
|
382
377
|
requested = os.environ.get("EMBED_BACKEND", "mlx").lower().strip()
|
|
383
378
|
custom_model = os.environ.get("EMBED_MODEL")
|
|
384
379
|
|
|
380
|
+
# Phase H: default selection may use the warm daemon. An explicit
|
|
381
|
+
# EMBED_BACKEND is a caller override and must select the requested
|
|
382
|
+
# in-process backend even when a daemon is already warm.
|
|
383
|
+
if not explicit_backend:
|
|
384
|
+
daemon = _probe_daemon()
|
|
385
|
+
if daemon is not None:
|
|
386
|
+
_BACKEND = daemon
|
|
387
|
+
return _BACKEND
|
|
388
|
+
|
|
385
389
|
if requested == "ollama":
|
|
386
390
|
_BACKEND = OllamaBackend(model=custom_model or OLLAMA_DEFAULT_MODEL)
|
|
387
391
|
return _BACKEND
|
|
@@ -257,7 +257,7 @@ def _cosine_rerank(conn, candidates: list[dict], goal_vec: list[float]) -> list[
|
|
|
257
257
|
ids = [r["id"] for r in candidates]
|
|
258
258
|
placeholders = ",".join("?" * len(ids))
|
|
259
259
|
emb_rows = conn.execute(
|
|
260
|
-
f"SELECT fact_id, vec FROM embeddings WHERE fact_id IN ({placeholders})",
|
|
260
|
+
f"SELECT fact_id, vec FROM embeddings WHERE fact_id IN ({placeholders})", # nosec: only ?-placeholders / constant fragments interpolated; values bound as params
|
|
261
261
|
ids,
|
|
262
262
|
).fetchall()
|
|
263
263
|
emb_map = {row["fact_id"]: _unpack_floats(row["vec"]) for row in emb_rows}
|
|
@@ -242,14 +242,14 @@ class SQLiteEdgesStore:
|
|
|
242
242
|
if direction in {"out", "both"}:
|
|
243
243
|
rows.extend(
|
|
244
244
|
self._conn.execute(
|
|
245
|
-
f"SELECT * FROM edges WHERE source = ?{rel_sql}",
|
|
245
|
+
f"SELECT * FROM edges WHERE source = ?{rel_sql}", # nosec: only ?-placeholders / constant fragments interpolated; values bound as params
|
|
246
246
|
(node_id, *rel_args),
|
|
247
247
|
).fetchall()
|
|
248
248
|
)
|
|
249
249
|
if direction in {"in", "both"}:
|
|
250
250
|
rows.extend(
|
|
251
251
|
self._conn.execute(
|
|
252
|
-
f"SELECT * FROM edges WHERE target = ?{rel_sql}",
|
|
252
|
+
f"SELECT * FROM edges WHERE target = ?{rel_sql}", # nosec: only ?-placeholders / constant fragments interpolated; values bound as params
|
|
253
253
|
(node_id, *rel_args),
|
|
254
254
|
).fetchall()
|
|
255
255
|
)
|
|
@@ -333,19 +333,19 @@ def _upsert_lesson(
|
|
|
333
333
|
with conn.cursor() as cur:
|
|
334
334
|
if project is None:
|
|
335
335
|
cur.execute(
|
|
336
|
-
f"DELETE FROM {schema}.semantic_facts "
|
|
336
|
+
f"DELETE FROM {schema}.semantic_facts " # nosec: schema is a validated identifier (^[a-z][a-z0-9_]*$); values bound as params
|
|
337
337
|
"WHERE subject = %s AND project IS NULL;",
|
|
338
338
|
(subject,),
|
|
339
339
|
)
|
|
340
340
|
else:
|
|
341
341
|
cur.execute(
|
|
342
|
-
f"DELETE FROM {schema}.semantic_facts "
|
|
342
|
+
f"DELETE FROM {schema}.semantic_facts " # nosec: schema is a validated identifier (^[a-z][a-z0-9_]*$); values bound as params
|
|
343
343
|
"WHERE subject = %s AND project = %s;",
|
|
344
344
|
(subject, project),
|
|
345
345
|
)
|
|
346
346
|
if vec_lit is None:
|
|
347
347
|
sql = (
|
|
348
|
-
f"INSERT INTO {schema}.semantic_facts "
|
|
348
|
+
f"INSERT INTO {schema}.semantic_facts " # nosec: schema is a validated identifier (^[a-z][a-z0-9_]*$); values bound as params
|
|
349
349
|
"(subject, predicate, object, confidence, status, embedding, metadata, "
|
|
350
350
|
" project, tool, task_category, files_touched, "
|
|
351
351
|
" confidence_source, domain) "
|
|
@@ -368,7 +368,7 @@ def _upsert_lesson(
|
|
|
368
368
|
)
|
|
369
369
|
else:
|
|
370
370
|
sql = (
|
|
371
|
-
f"INSERT INTO {schema}.semantic_facts "
|
|
371
|
+
f"INSERT INTO {schema}.semantic_facts " # nosec: schema is a validated identifier (^[a-z][a-z0-9_]*$); values bound as params
|
|
372
372
|
"(subject, predicate, object, confidence, status, embedding, metadata, "
|
|
373
373
|
" project, tool, task_category, files_touched, "
|
|
374
374
|
" confidence_source, domain) "
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: api-registry-bridge
|
|
3
|
-
description: Use when Phase 1 Assess or Phase 5 Iterate detects a new API dependency, API config fails, or the user asks to "register this API" or "check the API registry". Consults api-registry plugin; degrades gracefully if plugin is absent.
|
|
3
|
+
description: Use when Phase 1 Assess or Phase 5 Iterate detects a new API dependency, API config fails, or the user asks to "register this API" or "check the API registry". Consults api-registry plugin; degrades gracefully if plugin is absent. NOT for PRD strategic grounding (use `prd-bridge`) or UI verification (use `ibr-bridge`).
|
|
4
4
|
user-invocable: false
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -141,7 +141,7 @@ captures should set these explicitly when the signal is clear:
|
|
|
141
141
|
| Field | Skill default | Example | When to override |
|
|
142
142
|
|---|---|---|---|
|
|
143
143
|
| `--tool` | `claude-code` (Stop hook fires inside Claude Code) | `claude-code` | If the conversation references another tool authoring the decision (rare) |
|
|
144
|
-
| `--model` | `claude-opus-4-
|
|
144
|
+
| `--model` | `claude-opus-4-8` (or `$CLAUDE_MODEL`) | `claude-sonnet-5` | When you know the active sub-agent's model |
|
|
145
145
|
| `--author` | `auto` (skill-driven captures) | `auto` | Always `auto` for in-session skill writes |
|
|
146
146
|
| `--project` | derived from entity prefix or `$CLAUDE_PROJECT_DIR` | `build-loop` | Almost never; the default is right |
|
|
147
147
|
| `--task-category` | `unknown` if no signal | `bugfix`, `feature`, `refactor`, `research`, `docs`, `migration`, `experiment`, `config` | **Set explicitly** whenever the conversational signal is clear (user said "fix this bug" → `bugfix`; "add this feature" → `feature`; "investigate why X" → `research`) |
|
|
@@ -367,8 +367,9 @@ self-heals as more signal arrives.
|
|
|
367
367
|
|
|
368
368
|
## When the user runs `/knowledge:review`, surface…
|
|
369
369
|
|
|
370
|
-
The review surface (loaded by `build-loop:knowledge
|
|
371
|
-
`scripts/knowledge_review.py`)
|
|
370
|
+
The review surface (loaded by `build-loop:knowledge` review mode —
|
|
371
|
+
`references/review-mode.md`, backed by `scripts/knowledge_review.py`)
|
|
372
|
+
shows four sections of decisions and
|
|
372
373
|
procedures awaiting human attention:
|
|
373
374
|
|
|
374
375
|
1. **Review queue** — tier-3 / inferred captures sitting in
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: auto-finding-capture
|
|
3
|
+
description: Project-scoped skill documenting build-loop's DEFAULT-ON auto-capture of clearly-identified findings/issues into the backlog, regardless of which terminal or agent surfaced them. Provides the detection contract, routing rules, dedup strategy, and the standing rule that agents/critics must NOT gate identified issues behind a user selection. Use when an agent, audit, or critic surfaces a concrete severity-labeled issue, or when reasoning about where findings persist.
|
|
4
|
+
user-invocable: false
|
|
5
|
+
when_to_use: |
|
|
6
|
+
- Any agent/audit/critic states a concrete, severity-labeled issue in the session
|
|
7
|
+
- A dispatched ad-hoc audit (Codex, NavGator, a security pass) returns findings in conversation
|
|
8
|
+
- You are about to ask the user "which of these should I add to the backlog?" — DON'T; capture is automatic
|
|
9
|
+
- You need to know where session findings persist and how they dedup
|
|
10
|
+
namespace: .build-loop/backlog/ (review-queue overflow: .build-loop/proposals/)
|
|
11
|
+
companion_scripts:
|
|
12
|
+
- scripts/scan_findings/__main__.py — Stop-hook deterministic findings sweep
|
|
13
|
+
- scripts/scan_findings/detect.py — detector (severity-label + structured-JSON extraction)
|
|
14
|
+
- scripts/backlog.py — the single writer (new/sync/list); mirrors to build-loop-memory on sync
|
|
15
|
+
- scripts/review_finding_gate.py — canonical severity taxonomy (SEVERITY_MAP), reused for normalization
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
<!-- SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com> | SPDX-License-Identifier: Apache-2.0 -->
|
|
19
|
+
|
|
20
|
+
# auto-finding-capture — clearly-identified findings land in the backlog by DEFAULT
|
|
21
|
+
|
|
22
|
+
A clearly-identified finding is durable work. The user should never have to
|
|
23
|
+
remember to add it, and should never be asked to select which findings to keep.
|
|
24
|
+
Whenever ANY agent, audit, or critic surfaces a concrete issue in a session —
|
|
25
|
+
build-loop's own in-run reviewers OR an ad-hoc audit dispatched from a plain
|
|
26
|
+
terminal (Codex, NavGator, a one-off security pass) — that finding is captured
|
|
27
|
+
to the backlog automatically at session end.
|
|
28
|
+
|
|
29
|
+
## Standing rule (binding)
|
|
30
|
+
|
|
31
|
+
**Do not gate an identified issue behind a user selection.** If an agent or
|
|
32
|
+
critic has stated a concrete, severity-labeled finding, it is captured — full
|
|
33
|
+
stop. Asking "which of these should I add to the backlog?" is the workflow
|
|
34
|
+
violation this skill exists to remove. Surface findings in conversation AND let
|
|
35
|
+
them persist; the two are not in tension. The user can always drop a backlog
|
|
36
|
+
item later (`status: dropped` + `backlog.py sync`); they cannot recover a
|
|
37
|
+
finding that was never written down.
|
|
38
|
+
|
|
39
|
+
This mirrors the analogous DEFAULT-ON behavior for DECISIONS
|
|
40
|
+
(`auto-decision-capture` + `scan_transcript_for_decisions.py`) and for
|
|
41
|
+
CORRECTIONS/LESSONS (`scan_corrections`). Findings are the third lane.
|
|
42
|
+
|
|
43
|
+
## How it works — deterministic Stop-hook sweep
|
|
44
|
+
|
|
45
|
+
The Stop hook in `hooks/hooks.json` runs `python3 -m scan_findings` against
|
|
46
|
+
`$CLAUDE_TRANSCRIPT_PATH` at session end. It is **deterministic (zero-LLM)** —
|
|
47
|
+
pure regex + JSON parsing, no Ollama dependency — so it fails open with no model
|
|
48
|
+
installed. It scans the agent-authored surface of the transcript: host-assistant
|
|
49
|
+
text blocks AND `tool_result` blocks (where a dispatched sub-agent's condensed
|
|
50
|
+
return lands). It does NOT reach a sub-agent's isolated internal transcript —
|
|
51
|
+
only what surfaces in the session, which is exactly the gap this closes (ad-hoc
|
|
52
|
+
audits report their findings into the conversation).
|
|
53
|
+
|
|
54
|
+
### What counts as a clearly-identified finding (high precision over recall)
|
|
55
|
+
|
|
56
|
+
| Signal | Example | Route |
|
|
57
|
+
|---|---|---|
|
|
58
|
+
| **Structured findings JSON** (recognized severity) | `{"findings":[{"severity":"high","title":"Token logged in plaintext"}]}` | **backlog** |
|
|
59
|
+
| **Prose severity label** (UPPERCASE) + concrete clause | `HIGH: verify-install.yml interpolates dispatch input into shell — command injection` | **backlog** |
|
|
60
|
+
| **Explicit `severity:` field** + clause | `severity: medium — cache key collides across tenants` | **backlog** |
|
|
61
|
+
| Structured finding with **absent/unknown severity** | `{"severity":"banana","title":"..."}` | **review queue** |
|
|
62
|
+
| Prose **finding keyword / Bug:/Issue: prefix**, NO severity | `I suspect a race condition in the worker pool` | **review queue** |
|
|
63
|
+
| Anything else (questions, "low latency" prose, hedges) | `Is this high risk?` | ignored |
|
|
64
|
+
|
|
65
|
+
Routing is MECE: a **recognized severity** → backlog; a **finding signal without
|
|
66
|
+
a recognized severity** → `.build-loop/proposals/` for human triage; **neither**
|
|
67
|
+
→ ignored. A false backlog item is worse than a missed one, so anything short of
|
|
68
|
+
an asserted severity goes to the review queue, never straight to the backlog.
|
|
69
|
+
|
|
70
|
+
### Backlog mapping
|
|
71
|
+
|
|
72
|
+
Findings are written through `backlog.py new` (the single, host-agnostic writer
|
|
73
|
+
— never by hand), with:
|
|
74
|
+
|
|
75
|
+
- `--type fix`, `--area audit`
|
|
76
|
+
- `--priority` from severity: critical→P0, high→P1, medium→P2, low→P3
|
|
77
|
+
- `--provenance-source auto-finding-sweep[:<agent>]` — marks the auto-sweep and
|
|
78
|
+
the originating agent where detectable (e.g. `auto-finding-sweep:security-reviewer`)
|
|
79
|
+
- `--provenance-ref finding-hash:<hash>` — the cross-session dedup key
|
|
80
|
+
|
|
81
|
+
### Dedup (idempotent)
|
|
82
|
+
|
|
83
|
+
Each finding carries a stable `finding-hash` = sha1 of its severity-stripped
|
|
84
|
+
normalized clause. Before writing, the sweep loads every existing backlog item
|
|
85
|
+
(active + archived) and review proposal and collects their hashes + normalized
|
|
86
|
+
titles. A candidate is skipped when its hash already exists OR its normalized
|
|
87
|
+
title equals an existing open item's title. Re-running the sweep on the same
|
|
88
|
+
transcript creates nothing new. The same finding re-stated at a different
|
|
89
|
+
severity (or re-swept on a later Stop) hashes identically and is deduped.
|
|
90
|
+
|
|
91
|
+
## Hook safety contract
|
|
92
|
+
|
|
93
|
+
Non-blocking, fail-open, identical guardrails to the decision sweep:
|
|
94
|
+
|
|
95
|
+
- any error logs and exits 0 (the hook backgrounds with `nohup … &; printf '{}'`)
|
|
96
|
+
- `.build-loop/.no-capture` (per-session opt-out) → clean exit 0
|
|
97
|
+
- single-flight `fcntl.flock` on `/tmp/build-loop-findings-scan.lock`
|
|
98
|
+
- wall-clock budget `SCAN_FINDINGS_BUDGET_S` (default 15s); partial completion is
|
|
99
|
+
safe (each backlog write is its own atomic `backlog.py new` process)
|
|
100
|
+
- durable log at `${XDG_STATE_HOME:-~/.local/state}/build-loop/findings-scan.log`
|
|
101
|
+
|
|
102
|
+
## Manual run
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
# What the Stop hook runs (PYTHONPATH lets `-m scan_findings` resolve):
|
|
106
|
+
PYTHONPATH=scripts python3 -m scan_findings \
|
|
107
|
+
--workdir "$PWD" --transcript "$CLAUDE_TRANSCRIPT_PATH" --print-json
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
`--print-json` reports `{candidates, backlog, review, skipped_dup}`. After a
|
|
111
|
+
sweep, `python3 scripts/backlog.py sync --repo .` refreshes the INDEX and mirrors
|
|
112
|
+
the new items into `build-loop-memory/projects/<slug>/backlog/`.
|
|
113
|
+
|
|
114
|
+
## Deferred (NOT built)
|
|
115
|
+
|
|
116
|
+
An LLM-judged extraction path (for findings stated without a severity label or a
|
|
117
|
+
finding keyword) is a possible future extension. v1 is deterministic-only by
|
|
118
|
+
design: it satisfies the acceptance contract, adds no dependency, and keeps
|
|
119
|
+
precision high. Add the LLM path only against a named, observed miss in this repo.
|
|
@@ -14,17 +14,22 @@ A 5-phase development loop with a mandatory Phase 6: assess state and criteria,
|
|
|
14
14
|
|
|
15
15
|
## Routing
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
**`/build-loop:run` is the ONLY human-facing command.** Type it with any request in plain language — or just describe what you need — and the orchestrator classifies intent and routes to the right internal mode. No flags, no picking a mode, no other commands to learn. Everything below is reached by intent, not by a separate command; agents/build-loop invoke these internally.
|
|
18
18
|
|
|
19
|
-
|
|
19
|
+
Intent → internal mode:
|
|
20
20
|
|
|
21
|
-
- **Build** (default):
|
|
22
|
-
- **
|
|
23
|
-
- **
|
|
24
|
-
- **
|
|
25
|
-
- **Test**:
|
|
21
|
+
- **Build** (default): full 5-phase loop + mandatory Phase 6 Learn — implementation / fix / refactor / migrate / update / "add" / "wire up" language.
|
|
22
|
+
- **Debug**: deep iterative root-cause investigation — symptom language ("broken", "doesn't work", "failing"); also auto-invoked on Review-B failures.
|
|
23
|
+
- **Optimize**: metric-driven optimization loop — "speed up", "reduce", "improve" + a mechanical metric.
|
|
24
|
+
- **Research**: pre-decision analysis, research packet, no commits — "research", "evaluate", "compare", "should I".
|
|
25
|
+
- **Test**: static plugin-test suite — "test plugin", "validate plugin".
|
|
26
|
+
- **Root-cause analysis**: blameless RCA producing durable system levers — "root cause", "why did this fail", "post-mortem" (delegates to the `root-cause-analysis` skill).
|
|
27
|
+
- **Retrospective**: recursive learning retrospective on a build/project — "retrospective", "retro", "what did we learn", "review this project's trajectory" (delegates to `recursive-retrospective`). *Example: "I need a root cause and a retrospective" → run does both.*
|
|
28
|
+
- **Plan / spec**: "write a plan", "spec this" → spec-writing + plan-verify.
|
|
29
|
+
- **PRD**: "start a PRD", "spec out a new app" → prd-bridge / start-prd flow.
|
|
30
|
+
- **Self-improve / promote / knowledge / handoff / memory setup**: "scan recent runs", "promote this experiment", "record a decision", "hand this off", "set up memory" → the matching internal skill (self-improve, promotion-reviewer, knowledge, handoff, setup-memory).
|
|
26
31
|
|
|
27
|
-
|
|
32
|
+
**Design intent:** one command for humans, plain-language routing, everything else agent-invoked within build-loop. If a request doesn't match a mode, run treats it as a Build task or asks one clarifying question — it never makes the user pick a command.
|
|
28
33
|
|
|
29
34
|
### Parallelism config
|
|
30
35
|
|
|
@@ -95,6 +100,8 @@ Self-heal is **both reactive and proactive**. It is not only triggered by errors
|
|
|
95
100
|
|
|
96
101
|
**Investigate every open issue to root cause before declaring done — verified by a second subagent.** Before any "done"/completion claim, investigate EVERY open issue — failed tests, loose ends, errors, warnings, minor issues — none are left unaddressed. For each, reach the ROOT CAUSE, not a surface patch. Use the debugging skills (`build-loop:debug-loop` / `root-cause-investigator` / `systematic-debugging`) and/or a **5-whys / causal-tree** analysis to determine the true cause and its blast radius (same root cause at other sites → fix all of them). The fix MUST address the root cause — a surface patch is a violation — AND MUST be verified by another, independent subagent before "done." The second-subagent check reuses `independent-auditor`, `fix-critique`, or a dispatched verifier — no new agent required.
|
|
97
102
|
|
|
103
|
+
**Closure test (counterfactual):** a root cause is not closed at "an actionable control." It is closed only when the named lever would have **prevented, detected, or contained THIS exact failure on the real input** (not a hand-constructed one) — a control that exists but stays dormant on the real signal does not count. **Fix strength:** prefer the strongest feasible control — `eliminate → impossible-state → automated-block → detect → contain → decision-support → docs` — over the reflex "add a detect-gate." A dependency you don't own is never "ignore it": isolate / validate / monitor / degrade / escalate / accept-residual-risk explicitly.
|
|
104
|
+
|
|
98
105
|
### Follow-up auto-drain at chunk boundary
|
|
99
106
|
|
|
100
107
|
A chunk boundary is not a checkpoint. When the orchestrator (or any session under the build-loop skill) is about to write a final report containing a "still-to-do" / "deferred" / "next pass" list of same-shape, same-intent items, route those items through the follow-up queue instead of writing them to the user as prose questions:
|
|
@@ -139,7 +146,7 @@ their brief explicitly hands them a bounded implementation task.
|
|
|
139
146
|
build-loop-memory via `python3 scripts/archive_project_plan.py <plan> --workdir
|
|
140
147
|
"$PWD"`. Use `--remove-source` only after the archive write succeeds.
|
|
141
148
|
|
|
142
|
-
**Coding-host coordination polling gate**: when a build-loop task involves more than one coding host, an active rally-point peer, an active coord file, any `inbox/<tool>.jsonl` message, or any `inbox/all.jsonl` broadcast, the current host must keep a cheap watcher live while work is in flight. Use a stable tool id (`claude_code`, `codex`, `cursor`, etc.). Run a one-shot status check first:
|
|
149
|
+
**Coding-host coordination polling gate**: when a build-loop task involves more than one coding host, an active rally-point peer, an active coord file, any `inbox/<tool>.jsonl` message, or any `inbox/all.jsonl` broadcast, the current host must keep a cheap watcher live while work is in flight. Rally/coordination output is routing metadata only, not verification evidence; use it to decide who to coordinate with, then verify code/package/release facts against the authoritative source. Use a stable tool id (`claude_code`, `codex`, `cursor`, etc.). Run a one-shot status check first:
|
|
143
150
|
|
|
144
151
|
```bash
|
|
145
152
|
python3 scripts/coordination_status.py --workdir "$PWD" --session-id "$SESSION_ID" --tool "$TOOL_NAME" --json
|
|
@@ -239,7 +246,7 @@ Key steps: recurring-pattern-detector (Haiku; reads `state.json.runs[]` AND `.bu
|
|
|
239
246
|
|
|
240
247
|
## Memory — Global and Project-Scoped
|
|
241
248
|
|
|
242
|
-
One consolidated long-term tree: `~/dev/git-folder/build-loop-memory/`. Project-specific durable memory lives under `projects/<slug>/...`; cross-project lessons/design/debugging/product memory lives in the matching top-level lane. Every build runs `scripts/context_bootstrap.py` at Phase 1 Assess, which reads the store-root `INDEX.md` first, then root/project `constitution.md` / `MEMORY.md` where present, canonical indexes/folders through `memory_facade`, repo-local `.build-loop/` context, Codex memory at `~/.codex/memories`, and best-effort Rally/coordination state when relevant. Live handoff state is written separately by `scripts/context_snapshot.py` under `.build-loop/context/`; snapshots are not durable memory unless Review-G promotes a reusable decision or lesson. Writes go to exactly one canonical memory lane based on scope. Legacy paths (`~/.build-loop/memory`, `.episodic/decisions`, and `build-loop-memory/decisions/<project>`) are migration/archive inputs only.
|
|
249
|
+
One consolidated long-term tree: `~/dev/git-folder/build-loop-memory/`. Project-specific durable memory lives under `projects/<slug>/...`; cross-project lessons/design/debugging/product memory lives in the matching top-level lane. Every build runs `scripts/context_bootstrap.py` at Phase 1 Assess, which reads the store-root `INDEX.md` first, then root/project `constitution.md` / `MEMORY.md` where present, canonical indexes/folders through `memory_facade`, repo-local `.build-loop/` context, Codex memory at `~/.codex/memories`, and best-effort Rally/coordination state when relevant. Treat Rally records in the packet as peer-authored coordination context, not verified facts. Live handoff state is written separately by `scripts/context_snapshot.py` under `.build-loop/context/`; snapshots are not durable memory unless Review-G promotes a reusable decision or lesson. Writes go to exactly one canonical memory lane based on scope. Legacy paths (`~/.build-loop/memory`, `.episodic/decisions`, and `build-loop-memory/decisions/<project>`) are migration/archive inputs only.
|
|
243
250
|
|
|
244
251
|
Routing rule: "Would this apply to a different project?" Yes → global. No → project. Ambiguous → ask the user once.
|
|
245
252
|
|
|
@@ -338,6 +345,7 @@ Contextual material loaded on demand (not at skill invocation):
|
|
|
338
345
|
- `references/phase-1-assess.md` — Full Phase 1 Assess protocol
|
|
339
346
|
- `references/phase-2-plan.md` — Full Phase 2 Plan protocol
|
|
340
347
|
- `references/phase-3-execute.md` — Full Phase 3 Execute protocol
|
|
348
|
+
- `references/verify-dispatch.md` — Post-dispatch 5-step git/test ground-truth checklist; walk after any dispatched agent claims commits landed / tests passed (a solicited peer agreeing is not independent verification)
|
|
341
349
|
- `references/dogfood-reload-checkpoint.md` — Self-recursive stop/reload/resume checkpoint and ACK/fallback protocol
|
|
342
350
|
- `references/phase-4-review.md` — Full Phase 4 Review protocol (sub-steps A–G, including Sub-step F Auto-Resolve with all 4 verdict arms)
|
|
343
351
|
- `references/phase-5-iterate.md` — Full Phase 5 Iterate protocol
|
|
@@ -346,6 +354,7 @@ Contextual material loaded on demand (not at skill invocation):
|
|
|
346
354
|
- `references/leadership.md` — Initiative + decision-escalation doctrine (decide-at-70%, self-research → memory → peers → persona panel → human-only-for-irreversible, parallel-work-before-idling, token-posture gauge). Synthesized from intent-based leadership / mission command / two-door decisions.
|
|
347
355
|
- `references/research-trigger-policy.md` — Research plugin trigger/depth gate, t-shirt depth lower bounds, and final-claim citation/unavailable rule
|
|
348
356
|
- `references/task-capture-policy.md` — Read-only active task surface over existing plan/state/queue/backlog surfaces; no new task ledger by default
|
|
357
|
+
- `references/backlog-system.md` — Host-agnostic, multi-repo backlog system: MD+YAML items (canonical truth) + regenerable INDEX, pure-stdlib `scripts/backlog.py` (new/sync/list), one-way mirror to personal memory. Read via `BACKLOG.md`→`INDEX.md`→grep; write via the CLI
|
|
349
358
|
- `references/agent-role-taxonomy.md` — Lead/peer/coder-assessor/reviewer/skill responsibility map; use before adding or renaming agents.
|
|
350
359
|
- `references/capability-routing.md` — Full capability routing table, trigger conditions, sub-routers
|
|
351
360
|
- `references/recent-design-structures.md` — Recent UI structure library loaded by `design-contract-specialist` in Phase 2. Structures are options, not mandates.
|
|
@@ -418,7 +418,7 @@ Use the **6-Part Stack** for any system prompt or agent prompt:
|
|
|
418
418
|
Calibrate to model tier:
|
|
419
419
|
|
|
420
420
|
- **Frontier (T1 — Opus 4.6, GPT-5)**: can handle longer instructions, implicit reasoning, self-correction. Prefer clarity over verbosity.
|
|
421
|
-
- **Mid (T2 — Sonnet
|
|
421
|
+
- **Mid (T2 — Sonnet 5, GPT-5 class)**: explicit instructions; show, don't tell; 1-2 few-shot examples help.
|
|
422
422
|
- **Small/fast (T3 — Haiku 4.5, gpt-4-mini)**: keep prompts short; single task only; deterministic output format; more examples (3-5).
|
|
423
423
|
|
|
424
424
|
Review checklist — when auditing an existing prompt:
|
|
@@ -25,6 +25,55 @@ Phase 1 runs `node ${CLAUDE_PLUGIN_ROOT}/skills/build-loop/detect-plugins.mjs` a
|
|
|
25
25
|
| Modular systems pack | Phases 1-4 | Read `references/modular-systems-pack.md`; partition files/tasks MECE; prefer modular scalable boundaries unless an exception is documented |
|
|
26
26
|
| Codex subagent adapter | Phase 3 (Execute, Codex only) | Read `references/codex-subagents.md`; use `templates/codex-worker-prompt.md` for authorized Codex workers |
|
|
27
27
|
|
|
28
|
+
### Spec/Plan author router (intent-driven, ordered)
|
|
29
|
+
|
|
30
|
+
This router selects the single skill that AUTHORS the plan/spec/task graph for this run. It is a recommendation that DRIVES Phase 2 — recorded into `state.json.intent.spec_router`, consumed by Phase 2 Plan, never a hard block.
|
|
31
|
+
|
|
32
|
+
**Scope — author selection only.** `prd-bridge` is NOT in this router. It is mandatory in Phase 1 step 10 and only *consumes* an existing PRD (`docs/prd-*.md` → `state.json.prd`); it never authors. PRD-context is orthogonal to author selection — keep them separate.
|
|
33
|
+
|
|
34
|
+
**Governing rule**: select on intent + goal + context, not on a trigger-word match. A skill's word-overlap activation is not enough to call it (this is why `prd-builder` over-fired during build-loop runs). The router is the intent-side selection; it pairs with the activation-side guard (prd-builder's negative-trigger scope, which excludes in-flight changes and active build-loop runs).
|
|
35
|
+
|
|
36
|
+
Signals (all set during Phase 1 Assess):
|
|
37
|
+
|
|
38
|
+
- `run_active` — is a build-loop run active? (`true` for any `/build-loop:run` or build-orchestrator dispatch.)
|
|
39
|
+
- `plan_status` — `no-plan` | `plan-valid`, from the Phase 2 plan-exists gate (`.build-loop/plan.md` absent/empty + last `plan-verify` result).
|
|
40
|
+
- `intent_kind` — `PRD-author` | `build-plan` | `task-graph` | `none`, the LLM's read of what this run needs authored.
|
|
41
|
+
- `code_exists` — does the repo have substantive existing code? (Assess maps architecture: new/empty repo → false, existing repo → true.)
|
|
42
|
+
- `goal_scope` — `new-app` | `existing-app-change` | `in-build-task-breakdown`.
|
|
43
|
+
|
|
44
|
+
The rows are ORDERED — first match wins, so exactly one author is selected and no case matches two. The final default row makes the table exhaustive: every `(run_active, plan_status, intent_kind)` combination matches exactly one row, so nothing falls through.
|
|
45
|
+
|
|
46
|
+
| # | Match condition (first true wins) | route_type / action | Author selected | Why |
|
|
47
|
+
|---|---|---|---|---|
|
|
48
|
+
| 1 | `intent_kind == none` — nothing to author (Q&A, status, trivial, read-only, or advisory-context-only) | `noop` (call nothing) | — (`skill: null`) | no plan/spec/task graph needed |
|
|
49
|
+
| 2 | `run_active == false` AND greenfield PRD authoring is explicitly intended (new app with no code, OR `/start-prd` requested for a project) | `call` (or `recommend` outside an interactive session) | `prd-builder` | greenfield authoring + interactive intake; honors prd-builder's negative trigger — never fires inside an active run |
|
|
50
|
+
| 3 | `run_active == true` AND `plan_status == no-plan` | `call` | `build-loop:spec-writing` | author the in-build, non-interactive, gated plan for the orchestrator |
|
|
51
|
+
| 4 | `run_active == true` AND `plan_status == plan-valid` | `call` | `build-loop:writing-plans` (external) | turn the accepted plan into the task / dependency graph |
|
|
52
|
+
| 5 (default) | else (no earlier row matched) — the canonical case is `run_active == false` AND `intent_kind` is `build-plan` or `task-graph`: a direct, out-of-run invocation of plan authoring on an existing repo | `recommend` | `build-loop:spec-writing` | recommend, not auto-call: there is no active run to drive, so surface the author for the lead to run if it chooses |
|
|
53
|
+
|
|
54
|
+
The order is the contract: row 1 short-circuits before any author runs; row 2 is the only PRD author and is fenced off from active runs; rows 3 and 4 are mutually exclusive on `plan_status`; row 5 is the exhaustive `else` default that catches every remaining combination — its canonical case is an out-of-run `build-plan`/`task-graph` intent, but as the final row it also absorbs any residual (e.g. an out-of-run non-greenfield `PRD-author`) so no signal combination ever falls through. The table extends to any future author skill — add a row at the right precedence above the default, key it on the same signals, and the system still selects exactly one and records why.
|
|
55
|
+
|
|
56
|
+
`build-loop:writing-plans` (row 4) is the **external** `writing-plans` skill from the superpowers plugin — it is not vendored in this repo (`skills/writing-plans/SKILL.md` does not exist here). When it is absent, fall back to the inline "write a structured plan directly" path in the §"Core loop skills/assets" table above.
|
|
57
|
+
|
|
58
|
+
> Future refinement (do not over-build now): for monorepos, a `target_code_exists` signal (does the *target sub-path* already have code, vs the repo as a whole) would refine row 2's greenfield test. Out of scope until a monorepo case demands it.
|
|
59
|
+
|
|
60
|
+
**Phase 1 record (the consumable contract)**. Phase 1 writes the matched decision into `state.json.intent.spec_router` so a Codex lead (no `Skill()` auto-activation) can consume it deterministically:
|
|
61
|
+
|
|
62
|
+
```json
|
|
63
|
+
{
|
|
64
|
+
"route_type": "author",
|
|
65
|
+
"action": "call" | "recommend" | "noop",
|
|
66
|
+
"skill": "build-loop:spec-writing" | "build-loop:writing-plans" | "prd-builder" | null,
|
|
67
|
+
"fallback": "<fallbacks.md section or inline guidance> | null",
|
|
68
|
+
"matched_row": 1 | 2 | 3 | 4 | 5,
|
|
69
|
+
"signals": { "run_active": true, "plan_status": "no-plan", "intent_kind": "build-plan", "code_exists": true, "goal_scope": "existing-app-change" }
|
|
70
|
+
}
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
`action: "noop"` → `skill: null`, Phase 2 authors nothing from the router (terminal). `action: "recommend"` → `skill` carries the recommended author name (so the lead knows what to run if it chooses); Phase 2 surfaces the recommendation in the report and does NOT auto-call. `action: "call"` → `skill` is non-null and Phase 2 calls it. The three actions are distinct: `call` auto-invokes, `recommend` surfaces-only, `noop` skips.
|
|
74
|
+
|
|
75
|
+
**Phase 2 read (the load-bearing wire)**. Phase 2 Plan READS `state.json.intent.spec_router` and acts on `action`/`skill` — it does NOT independently re-decide which author to call. See `references/phase-2-plan.md` step 0.
|
|
76
|
+
|
|
28
77
|
### Phase quick reference
|
|
29
78
|
|
|
30
79
|
| # | Phase | Purpose | Sub-steps / key actions |
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
<!-- SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com> | SPDX-License-Identifier: Apache-2.0 -->
|
|
2
|
+
<!-- CONFIG: v1.0 | T2 | plugin(report-writeup) | Analytical | SCORE: 22/25 [A:5|C:4|Cs:5|D:4|Cp:4] -->
|
|
3
|
+
|
|
4
|
+
# Experiment / data-analysis results — reusable template + guide
|
|
5
|
+
|
|
6
|
+
One write-up shape for any analysis: A/B, DOE / full-factorial / fractional-factorial,
|
|
7
|
+
regression, backtest, ablation, regression-test. Fill the blanks; delete the Method
|
|
8
|
+
sub-lines that don't apply; never delete the honesty rails (n, direction legend,
|
|
9
|
+
certainty, "not computed — why").
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## The template (copy below this line)
|
|
14
|
+
|
|
15
|
+
```markdown
|
|
16
|
+
# <test name> — results
|
|
17
|
+
<!-- headline: DECISION: <build X / don't / inconclusive> · certainty: <high|medium|low> · n=<N> -->
|
|
18
|
+
|
|
19
|
+
## Objective
|
|
20
|
+
- **Testing:** <the one thing under test>
|
|
21
|
+
- **Why:** <the recurring problem or the decision this informs>
|
|
22
|
+
- **Payoff if it succeeds (user-visible):** <what changes for the end user / consumer>
|
|
23
|
+
- **Decision this unblocks:** <what we do differently on a pass vs a fail>
|
|
24
|
+
|
|
25
|
+
## What we did
|
|
26
|
+
- **Analysis type:** <A/B | DOE full-factorial | fractional-factorial | regression | backtest | ablation | regression-test | other>
|
|
27
|
+
- **Method (1 paragraph):** <how it ran, start to finish>
|
|
28
|
+
- **Sample size n = <N>** (<unit: bugs / runs / requests / rows / sessions>) ← never omit
|
|
29
|
+
- **Arms / factors** (keep the line that fits your type):
|
|
30
|
+
- A/B: control = <...> · treatment(s) = <...>
|
|
31
|
+
- DOE/factorial: factors × levels = <...> · runs = <...> · design = <full | fractional res IV | ...>
|
|
32
|
+
- Regression: predictors = <...> · response = <...> · model = <OLS | logistic | ...>
|
|
33
|
+
- Backtest: data + window = <...> · in/out-of-sample split = <...>
|
|
34
|
+
- Ablation: component removed = <...> · baseline = <...>
|
|
35
|
+
- **Held constant (confounds managed):** <model, environment, inputs, seed, …>
|
|
36
|
+
- **Measurement:** <who/what scored, against what ground truth, and exactly how>
|
|
37
|
+
- Blinding: <none | labels withheld | fully blinded> — <if you claim blinded, say what made it so; if a tell leaked, say so>
|
|
38
|
+
|
|
39
|
+
## Hypothesis
|
|
40
|
+
- **Predicted:** <outcome> **because** <mechanism / prior reason>
|
|
41
|
+
- **Pass means:** <what it implies> → **action:** <...>
|
|
42
|
+
- **Fail means:** <what it implies> → **action:** <...>
|
|
43
|
+
- **Pre-registered?** <yes + link | no> · decision rule fixed before running? <yes | no>
|
|
44
|
+
|
|
45
|
+
## Results
|
|
46
|
+
|
|
47
|
+
**Metric legend — state direction for EVERY metric (this is mandatory):**
|
|
48
|
+
| Metric | Measures | Direction | Scale |
|
|
49
|
+
|--------|----------|-----------|-------|
|
|
50
|
+
| <m1> | <...> | higher better / lower better | <0–2 · % · ms · count · $> |
|
|
51
|
+
| <cost> | effort/latency/$ | lower better (only counts when quality ties or wins) | <calls · s · $> |
|
|
52
|
+
|
|
53
|
+
**Data:**
|
|
54
|
+
| <arm / run / config> | <m1> | <m2> | <cost> |
|
|
55
|
+
|---|---|---|---|
|
|
56
|
+
| <...> | | | |
|
|
57
|
+
|
|
58
|
+
**Statistics — fill what you actually computed; for the rest write "not computed — <why>":**
|
|
59
|
+
- **Effect size:** <metric: value (e.g. Δ, Cohen's d, lift %)> | not computed — <why>
|
|
60
|
+
- **Statistical significance:** <test, p-value, CI> | **NOT COMPUTABLE — n=<N> too small (directional only)**
|
|
61
|
+
- **Correlation:** <vars: r> | not computed — <why>
|
|
62
|
+
- **Goodness of fit** (regression only): <R², adj-R², residual check> | n/a
|
|
63
|
+
- **Winner / direction:** <which, on which metrics>
|
|
64
|
+
|
|
65
|
+
## Interpretation
|
|
66
|
+
- **Certainty:** ✅ high | ⚠️ medium | ❓ low — <why, tied to n + method strength>
|
|
67
|
+
- **What it shows / does NOT show:** <scope of the claim>
|
|
68
|
+
- **Threats to validity / confounds:** <weak blinding (how), small n, self-selected subject, model substitution, single grader, …>
|
|
69
|
+
- **Unknowns:** <what could not be verified>
|
|
70
|
+
- **Honesty check:** every adjective here is earned by the method — no "fair / blind / significant / proven" unless the method above supports it.
|
|
71
|
+
|
|
72
|
+
## Next steps
|
|
73
|
+
- [ ] **Implement winner:** <...> (or: do not implement — <why>)
|
|
74
|
+
- [ ] **Stronger re-test:** <what raises certainty — larger n, cleaner blinding, a real significance test, second grader>
|
|
75
|
+
- [ ] **New tests suggested:** <...>
|
|
76
|
+
- [ ] **Risks to monitor after rollout:** <...>
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## Usage guide
|
|
82
|
+
|
|
83
|
+
1. **Pick one Method sub-line** for your analysis type; delete the others. The honesty
|
|
84
|
+
rails — `n`, the direction legend, the statistics block, certainty — stay regardless
|
|
85
|
+
of type.
|
|
86
|
+
2. **Direction legend is not optional.** A table of numbers is unreadable without "higher
|
|
87
|
+
or lower better" per metric. Cost metrics (calls, time, $) are lower-better but only
|
|
88
|
+
count as a win when quality ties or improves (Accuracy > Speed > Cost).
|
|
89
|
+
3. **When stats don't apply, say so explicitly** — never leave significance/correlation/
|
|
90
|
+
fit blank or imply them. Rules of thumb: significance/correlation need roughly n ≥ 8–10+
|
|
91
|
+
to mean anything; below that write "directional only (n too small)". Regression fit
|
|
92
|
+
(R²) is n/a unless you actually fit a model. DOE reports main effects + interactions,
|
|
93
|
+
not p-values, unless replicated.
|
|
94
|
+
4. **Earn every adjective.** Do not write "fair", "blind", "robust", "significant", or
|
|
95
|
+
"proven" unless the Method section shows what made it so. If blinding leaked (e.g. the
|
|
96
|
+
treatment's output format was a tell), record it as a threat to validity — don't claim
|
|
97
|
+
blindness.
|
|
98
|
+
5. **Headline = decision + certainty + n.** A decision-maker should get "build it / don't /
|
|
99
|
+
inconclusive", how sure, and on how much data, from the first line.
|
|
100
|
+
6. **Pre-register when the result will drive a real decision** — commit Objective +
|
|
101
|
+
Hypothesis + decision rule before running, so the verdict can't be retrofit.
|
|
@@ -136,6 +136,20 @@ python3 ${CLAUDE_PLUGIN_ROOT}/scripts/memory_writer.py \
|
|
|
136
136
|
|
|
137
137
|
The writer auto-detects `source_repo` from the workdir's git remote, appends a row to the lane-local `INDEX.jsonl`, appends a row to the global update ledger at `indexes/updates.jsonl`, and (on update) preserves `created_at` + `applied_in_repos` so cross-repo validation history survives edits. Direct writes are repair/fallback work only: use them only when no canonical writer exists or the current task is explicitly a memory-system repair, then run the host index/check step and verify reachability.
|
|
138
138
|
|
|
139
|
+
### Codex cross-repo write guard
|
|
140
|
+
|
|
141
|
+
When using Codex `apply_patch`, path resolution is relative to the active
|
|
142
|
+
workspace, not to a shell command `workdir`. If a memory artifact belongs in a
|
|
143
|
+
different repo such as `build-loop-memory`, use one of these safe paths:
|
|
144
|
+
|
|
145
|
+
- Prefer `scripts/memory_writer.py` for normal memory writes.
|
|
146
|
+
- If direct repair is unavoidable, pass absolute target paths to `apply_patch`.
|
|
147
|
+
- Before moving a generated file that already appeared in a Codex file card,
|
|
148
|
+
leave an openable pointer, mirror, or stub at the old path and update any
|
|
149
|
+
dependent indexes/links. Do not strand UI/Finder file cards at missing paths.
|
|
150
|
+
- After a move, verify both the canonical destination and every old path that a
|
|
151
|
+
user-facing file card, index, or markdown link still references.
|
|
152
|
+
|
|
139
153
|
### Reader side — surface peer writes via INDEX.jsonl
|
|
140
154
|
|
|
141
155
|
Between phases (or at every M2 heartbeat), tail since your last check:
|