baldart 4.99.0 → 5.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +114 -0
- package/README.md +1 -0
- package/VERSION +1 -1
- package/framework/.claude/agents/CHANGELOG.md +105 -0
- package/framework/.claude/agents/REGISTRY.md +3 -1
- package/framework/.claude/agents/code-reviewer.md +84 -400
- package/framework/.claude/agents/codebase-architect.md +71 -356
- package/framework/.claude/agents/coder.md +130 -291
- package/framework/.claude/agents/doc-reviewer.md +127 -450
- package/framework/.claude/agents/plan-auditor.md +137 -436
- package/framework/.claude/agents/prd-card-writer.md +1 -1
- package/framework/.claude/agents/qa-sentinel.md +59 -313
- package/framework/.claude/agents/skill-improver.md +60 -4
- package/framework/.claude/agents/ui-expert.md +104 -591
- package/framework/.claude/commands/codexreview.md +26 -4
- package/framework/.claude/hooks/agent-discovery-gate.js +2 -2
- package/framework/.claude/hooks/agent-discovery-info.sh +1 -0
- package/framework/.claude/skills/new/CHANGELOG.md +39 -0
- package/framework/.claude/skills/new/SKILL.md +1 -1
- package/framework/.claude/skills/new/references/codex-gate.md +25 -11
- package/framework/.claude/skills/new/references/implement.md +49 -5
- package/framework/.claude/skills/new/references/review-cycle.md +20 -5
- package/framework/.claude/skills/new/scripts/build-review-bundle.mjs +112 -0
- package/framework/.claude/skills/new/scripts/doc-invariants.mjs +166 -0
- package/framework/.claude/skills/new2/CHANGELOG.md +13 -0
- package/framework/.claude/skills/new2/SKILL.md +5 -1
- package/framework/.claude/workflows/new-card-review.js +16 -2
- package/framework/.claude/workflows/new2-resolve.js +11 -4
- package/framework/.claude/workflows/new2.js +51 -3
- package/framework/agents/agent-operating-protocol.md +152 -0
- package/framework/agents/coding-standards.md +2 -2
- package/framework/agents/doc-audit-protocol.md +232 -0
- package/framework/agents/index.md +4 -0
- package/framework/agents/review-protocol.md +207 -0
- package/framework/docs/UPGRADE-3.12-UI-COHERENCE.md +1 -1
- package/framework/routines/finding-mine.routine.yml +56 -0
- package/framework/routines/index.yml +11 -0
- package/package.json +1 -1
- package/src/commands/configure.js +15 -0
- package/src/commands/doctor.js +69 -2
- package/src/utils/agent-slots.js +109 -0
- package/src/utils/overlay-merger.js +17 -8
- package/src/utils/symlinks.js +93 -33
|
@@ -0,0 +1,232 @@
|
|
|
1
|
+
# Doc Audit Protocol — doc-reviewer's audit-mode procedures
|
|
2
|
+
|
|
3
|
+
**Purpose**: `doc-reviewer` runs in two very different shapes: the **per-card
|
|
4
|
+
pass** (241 spawns/month measured — scope-guarded, invariant-driven, fast) and
|
|
5
|
+
the **audit mode** (nightly `doc-review` routine, weekly `full-sweep`, manual
|
|
6
|
+
general audits). The audit procedures — drift validator suite, topological
|
|
7
|
+
generation, epistemic metadata, SCIP anchors, coverage gauges — were paid by
|
|
8
|
+
EVERY per-card spawn while applying to a tiny fraction of runs. This module is
|
|
9
|
+
their home (v5.0.0): `doc-reviewer` per-card core stays lean and cites this
|
|
10
|
+
module ONLY when invoked without card context (audit mode). On divergence,
|
|
11
|
+
**this module wins**.
|
|
12
|
+
|
|
13
|
+
**Consumers**: `doc-reviewer` (audit mode — nightly/weekly routines, general
|
|
14
|
+
audits), `wiki-curator` (coverage signals), the `full-sweep` routine.
|
|
15
|
+
|
|
16
|
+
## Contract
|
|
17
|
+
|
|
18
|
+
- **Dispatch**: `agents/doc-audit-protocol.md SECTION=<audit-process|drift-validators|topological-generation|epistemic-metadata|scip-refs|schema-drift|coverage-gauges>`.
|
|
19
|
+
Grep for `### SECTION: <name>` and Read ONLY the sections your run needs.
|
|
20
|
+
- **When**: audit mode = invoked WITHOUT card context (general audit, nightly
|
|
21
|
+
run, cleanup) or explicitly told "full audit". A per-card invocation does NOT
|
|
22
|
+
read this module wholesale — the ONLY exceptions are the sections the agent
|
|
23
|
+
core CITES explicitly (v5.0.1): `epistemic-metadata` + `scip-refs` (ADR
|
|
24
|
+
quantitative claims), `topological-generation` (a per-card pass writing MORE
|
|
25
|
+
than one doc file), `schema-drift` (a documented payload's field SET changed
|
|
26
|
+
in the diff). Everything else (audit-process, drift-validators,
|
|
27
|
+
coverage-gauges) stays audit-only.
|
|
28
|
+
- **Graceful degradation (MANDATORY)**: every `npm run validate:*` /
|
|
29
|
+
`scripts/*.mjs` referenced here is a project-specific example — check the
|
|
30
|
+
project's `package.json` first, run only what exists, and skip each absent
|
|
31
|
+
one with a one-line note. A validator only gates the audit when it both
|
|
32
|
+
exists AND exits non-zero per its row.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
### SECTION: audit-process
|
|
37
|
+
|
|
38
|
+
General documentation review (no card scope):
|
|
39
|
+
|
|
40
|
+
1. **Map**: build a tree of all docs with purpose annotations.
|
|
41
|
+
2. **Risk scan**: identify the top 10 readability/bloat issues.
|
|
42
|
+
3. **Consistency check**: flag cross-doc conflicts and duplicated truths.
|
|
43
|
+
4. **Structure plan**: propose TOCs, indexes, cross-link improvements.
|
|
44
|
+
5. **Edit samples**: show concrete before/after (keep short).
|
|
45
|
+
|
|
46
|
+
Then run the drift validators (below) and integrate their findings before
|
|
47
|
+
declaring any verdict.
|
|
48
|
+
|
|
49
|
+
### SECTION: drift-validators
|
|
50
|
+
|
|
51
|
+
Run whichever of these validators the project actually ships (graceful
|
|
52
|
+
degradation per the Contract above). Outputs land in `docs/reports/*-drift.md`.
|
|
53
|
+
|
|
54
|
+
| Script | Detects | Exit on drift |
|
|
55
|
+
|---|---|---|
|
|
56
|
+
| `npm run validate:errors` | API error codes used in code but missing in `${paths.references_dir}/errors.md` | 1 if ≥1% missing |
|
|
57
|
+
| `npm run validate:perms` | Permission strings used in code but not defined in the project's permissions constants | 1 if any (security gap) |
|
|
58
|
+
| `npm run validate:env` | `process.env.*` references not in `${paths.references_dir}/env-vars.md` | 0 (advisory) |
|
|
59
|
+
| `npm run validate:frontmatter` | Missing required frontmatter fields (esp. ADR `status:`) | 1 if ADR missing status |
|
|
60
|
+
| `npm run validate:imports` | Banned chart libs imported outside allowlist | 1 if any |
|
|
61
|
+
| `npm run doctest` | Runnable `js` examples in API refs that throw at execution | 1 if any block fails; 0 if zero `js` blocks |
|
|
62
|
+
| `npm run regenerate:field-registry` | Field-registry drift vs current TS types | 0 (rewrites in place) |
|
|
63
|
+
| `npm run audit:full-sweep` | Weekly: aggregates all above + history delta | 0 always |
|
|
64
|
+
|
|
65
|
+
**Integrate findings**: BLOCKER exits → the audit MUST NOT declare PASS until
|
|
66
|
+
resolved. Advisory exits → include under a "Drift Watch" section of the
|
|
67
|
+
deliverable.
|
|
68
|
+
|
|
69
|
+
### SECTION: topological-generation
|
|
70
|
+
|
|
71
|
+
**Dependency-Aware Generation** — when writing or updating docs that span
|
|
72
|
+
multiple related files (an API module covering 5 routes, a data-model section
|
|
73
|
+
covering 3 collections, a feature touching API + lib + components), generate
|
|
74
|
+
documentation in **dependency-topological order**, never arbitrary order.
|
|
75
|
+
|
|
76
|
+
**Why**: DocAgent (ACL 2025, arXiv:2504.08725) reports that documenting
|
|
77
|
+
dependencies before their dependents lifted truthfulness from 61% → 95.74%
|
|
78
|
+
(+34.74pp absolute; topological ordering alone +7.89pp over a context-aware
|
|
79
|
+
baseline). When a doc references symbols whose definitions are not yet
|
|
80
|
+
documented, the writer hallucinates signatures, invents fields, and describes
|
|
81
|
+
behavior that does not match the implementation.
|
|
82
|
+
|
|
83
|
+
**Protocol**:
|
|
84
|
+
1. **Refresh the graph**: `npm run graph:doc-deps` → writes
|
|
85
|
+
`docs/reports/doc-dependency-graph.json` (`nodes`, `edges`, `cycles`,
|
|
86
|
+
`topological_order`, flat `known_identifiers` vocabulary).
|
|
87
|
+
2. **Sort your work** by `topological_order`: modules with no internal
|
|
88
|
+
dependencies first; route handlers that consume many libs last. When the
|
|
89
|
+
work touches files A, B, C: intersect `{A,B,C}` with the order and process
|
|
90
|
+
the intersection in that order.
|
|
91
|
+
3. **Honour the constraint**: a doc may only reference symbols that are (a)
|
|
92
|
+
already documented in a previously-finalized doc, or (b) present in
|
|
93
|
+
`known_identifiers`. A symbol that exists in neither is signal of a doc
|
|
94
|
+
hallucination or a missing implementation — flag it, never paper over it.
|
|
95
|
+
4. **Cycles**: SCC members are adjacent in `topological_order` — document a
|
|
96
|
+
whole cycle in one pass, never split it.
|
|
97
|
+
5. **Verify each doc**: `npm run validate:doc-symbols -- --doc <path>` (exit 1
|
|
98
|
+
when unknown-symbol ratio > 5%). Resolve by fixing typos, replacing
|
|
99
|
+
speculative names with real ones, or regenerating the graph.
|
|
100
|
+
6. **Audit-wide**: `npm run validate:doc-symbols -- --all` (nightly / pre-PR;
|
|
101
|
+
the weekly full-sweep runs it automatically).
|
|
102
|
+
|
|
103
|
+
This complements the drift validators: `validate-doc-symbols` gates the prose
|
|
104
|
+
itself, the validators gate registry completeness.
|
|
105
|
+
|
|
106
|
+
### SECTION: epistemic-metadata
|
|
107
|
+
|
|
108
|
+
When writing or reviewing an ADR (or any decision doc) with **quantitative
|
|
109
|
+
claims** — performance numbers, capacity limits, A/B results, latency targets,
|
|
110
|
+
cost figures, hit-rates, accuracy — include an `evidence[]` array in the
|
|
111
|
+
frontmatter.
|
|
112
|
+
|
|
113
|
+
**Why**: a retrospective audit of 62 architectural decisions
|
|
114
|
+
(arXiv 2601.21116) found 23% of ADR evidence stale within 2 months, 86% of it
|
|
115
|
+
discovered only REACTIVELY during incidents. `evidence[]` declares an explicit
|
|
116
|
+
trust horizon per claim; the nightly sweep flags `EVIDENCE_EXPIRED` /
|
|
117
|
+
`EVIDENCE_EXPIRING_SOON` proactively in
|
|
118
|
+
`docs/reports/frontmatter-evidence-currency.md`.
|
|
119
|
+
|
|
120
|
+
**When**: Always — ADR contains quantitative claims. Optional — categorical
|
|
121
|
+
decision resting on a measurable property ("provider X is 38% cheaper").
|
|
122
|
+
Never — purely structural decisions (file layout, naming, taxonomy).
|
|
123
|
+
|
|
124
|
+
**Template**:
|
|
125
|
+
```yaml
|
|
126
|
+
evidence:
|
|
127
|
+
- claim: "<short factual statement; one line>"
|
|
128
|
+
source: "<card ID, telemetry name, ADR ref, or 'vendor docs <date>'>"
|
|
129
|
+
observed_at: 'YYYY-MM-DD'
|
|
130
|
+
expires_at: 'YYYY-MM-DD' # default: observed_at + 6 months
|
|
131
|
+
confidence: high | medium | low
|
|
132
|
+
method: experiment | observation | analysis | external-reference
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
**`expires_at` defaults**: observed_at + 6 months; shorten to 3 months when the
|
|
136
|
+
metric is traffic-elastic (response times, hit-rates, queue depth) or rests on
|
|
137
|
+
vendor pricing/policy; 12 months for regulatory interpretations.
|
|
138
|
+
|
|
139
|
+
**Confidence rubric**: `high` = replicated measurement ≤3 months old, stable
|
|
140
|
+
under current load · `medium` = single measurement OR ageing (3–9 months) OR
|
|
141
|
+
moderate volatility · `low` = external quote, vendor claim, or >9 months
|
|
142
|
+
unverified.
|
|
143
|
+
|
|
144
|
+
**Method**: `experiment` (A/B, controlled benchmark) · `observation`
|
|
145
|
+
(production telemetry, log aggregation) · `analysis` (static analysis, cost
|
|
146
|
+
model) · `external-reference` (vendor docs, public benchmark).
|
|
147
|
+
|
|
148
|
+
**Reviewer checklist** (ADR completeness):
|
|
149
|
+
1. Does Context/Rationale/Consequences/Implementation contain a number?
|
|
150
|
+
2. If yes: is each number traced to an `evidence[]` entry with `expires_at`?
|
|
151
|
+
3. Are `observed_at` dates plausible (not future, not before `last_updated`)?
|
|
152
|
+
4. Is `confidence` calibrated (a vendor cost quote is rarely `high`)?
|
|
153
|
+
|
|
154
|
+
Run `node scripts/validate-frontmatter.mjs --evidence-only` before finalizing
|
|
155
|
+
(advisory, exit 0). See `${paths.references_dir}/frontmatter-standard.md § Optional
|
|
156
|
+
epistemic metadata`.
|
|
157
|
+
|
|
158
|
+
### SECTION: scip-refs
|
|
159
|
+
|
|
160
|
+
When documenting a **code-tied fact** — a permission helper, auth wrapper, API
|
|
161
|
+
route handler, scoring primitive, or any function whose identity matters to
|
|
162
|
+
the doc's correctness — you SHOULD anchor the prose to a compiler-stable SCIP
|
|
163
|
+
symbol ID via a `code_refs:` frontmatter entry.
|
|
164
|
+
|
|
165
|
+
**Why**: prose paths (`src/lib/<module>.ts#<symbol>`) rot silently on rename.
|
|
166
|
+
SCIP symbol IDs survive moves/renames as long as the index is rebuilt —
|
|
167
|
+
F1=80.4% link survival vs ~66% with path-only refs (arXiv 2506.16440).
|
|
168
|
+
|
|
169
|
+
**Frontmatter contract** (full spec:
|
|
170
|
+
`${paths.references_dir}/frontmatter-standard.md § Code References`):
|
|
171
|
+
```yaml
|
|
172
|
+
code_refs:
|
|
173
|
+
- symbol: 'src/lib/permissions/middleware.ts#checkPermission()'
|
|
174
|
+
scip_id: 'scip-typescript npm <project> <version> src/lib/permissions/`middleware.ts`/checkPermission().'
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
**Generate**: `npm run scip:index` then
|
|
178
|
+
`node scripts/scip-symbol-resolver.mjs --query <symbol>` → copy the id.
|
|
179
|
+
**Validate**: `node scripts/validate-scip-refs.mjs` (exit 1 on unresolved id;
|
|
180
|
+
the weekly full-sweep runs it). Refresh the symbol index with
|
|
181
|
+
`npm run scip:symbols` after codebase changes.
|
|
182
|
+
|
|
183
|
+
**Selection rule**: anchor the 2–4 most important symbols per doc — identity
|
|
184
|
+
anchoring, not exhaustive enumeration. For wider surfaces, link the module in
|
|
185
|
+
prose and anchor only the entry points.
|
|
186
|
+
|
|
187
|
+
**Cadence**: `index.scip` is gitignored; generation is on-demand for authoring
|
|
188
|
+
and automatic in the weekly sweep.
|
|
189
|
+
|
|
190
|
+
### SECTION: schema-drift
|
|
191
|
+
|
|
192
|
+
Beyond route existence, reconcile SCHEMAS and REGISTRIES against code:
|
|
193
|
+
|
|
194
|
+
**Field-level drift** — when a route's request/response schema (Zod or TS
|
|
195
|
+
types) changes, compare to `${paths.references_dir}/api/<domain>.md` and
|
|
196
|
+
`api/schemas.md`:
|
|
197
|
+
- diverged shape → `SCHEMA_DRIFT`
|
|
198
|
+
- new field in a documented payload → NEEDS_UPDATE
|
|
199
|
+
- removed field still documented → `ORPHAN_FIELD`
|
|
200
|
+
|
|
201
|
+
**Canonical API registries** (when `features.has_api_docs: true`) — you OWN
|
|
202
|
+
keeping the two canonical registries in sync with code on the nightly run (and
|
|
203
|
+
on any API-doc rewrite):
|
|
204
|
+
- **`${paths.api_errors}`** (errors.md) — the single registry of stable error
|
|
205
|
+
codes. A code added/renamed/removed in code → apply it to the right domain
|
|
206
|
+
section (trivial fix) or flag `DOC_GAP`. Endpoint docs link the domain
|
|
207
|
+
anchor, never duplicate the table.
|
|
208
|
+
- **`${paths.api_schemas}`** (schemas.md) — the documented projection of the
|
|
209
|
+
Zod/TS shapes (code stays the SSOT). Shape change → update the table or flag
|
|
210
|
+
`SCHEMA_DRIFT`.
|
|
211
|
+
Skip silently when either path is unset. This is the code→doc direction; the
|
|
212
|
+
`doc-writing-for-rag` skill handles doc-authoring (APPEND, no duplication).
|
|
213
|
+
|
|
214
|
+
**Endpoint count reconciliation** (file-system route handlers only, e.g.
|
|
215
|
+
Next.js App Router `route.ts` — adapt paths to the project):
|
|
216
|
+
- File count: `find <api-root> -name route.ts | wc -l`
|
|
217
|
+
- Endpoint count: `grep -rE '^export async function (GET|POST|PUT|PATCH|DELETE)' <api-root> | wc -l`
|
|
218
|
+
(a handler exporting GET + POST counts as 2)
|
|
219
|
+
- Reconcile against the `api/index.md` total; >5% divergence →
|
|
220
|
+
`ENDPOINT_COUNT_MISMATCH`. No file-system handlers → skip with a one-line
|
|
221
|
+
note.
|
|
222
|
+
|
|
223
|
+
### SECTION: coverage-gauges
|
|
224
|
+
|
|
225
|
+
Track per-dimension coverage over time; append the nightly metric to
|
|
226
|
+
`docs/reports/coverage-history.json` (the weekly full-sweep updates the
|
|
227
|
+
gauge):
|
|
228
|
+
- D01 API: endpoints documented / endpoints emitted
|
|
229
|
+
- D02 Data: collections documented / collections used
|
|
230
|
+
- D04 DS: components documented / reusable components in src/
|
|
231
|
+
- D09 Permissions: permissions documented / permissions used
|
|
232
|
+
- D10 Errors: error codes documented / error codes emitted
|
|
@@ -41,6 +41,7 @@ Route agents to the right module with minimal reading.
|
|
|
41
41
|
- If touching monitoring/logging -> read `agents/observability.md`.
|
|
42
42
|
- If tuning reasoning depth — setting a skill's `effort:` baseline or honoring an inline `effort=<level>` override -> read `agents/effort-protocol.md`.
|
|
43
43
|
- If authoring/auditing a subagent whose return would flood the orchestrator context (free-form analysis/audit/research) -> read `agents/return-contract-protocol.md` and bound its final message: COMPACT headline + `path:line` findings + disk pointer, persist the long form. The input-side twin of the effort protocol.
|
|
44
|
+
- If authoring/editing an agent definition -> the shared operating procedures (injection guard, retrieval, memory, tool budget) live in `agents/agent-operating-protocol.md` and the reviewer verification passes (challenge/simulation/CoVe/risk) in `agents/review-protocol.md` — cite their sections, never re-inline them; keep only the 1-line binding versions in the agent body.
|
|
44
45
|
- If a skill performs runtime-mechanical operations (spawn an agent, gate on the user, track state, accelerate a fan-out, run an adversarial review) and must work on BOTH Claude Code and Codex -> read `agents/runtime-portability-protocol.md`: bind each abstract operation to the runtime (Claude `Task`/`Workflow`/`AskUserQuestion`/task-spine ↔ Codex named-agent spawn / inline / plain prompt / file-backed queue), detect the runtime once at kickoff, never probe Claude-only primitives on Codex. Shared policy (delegation, no-silent-fallback, worktree isolation) stays in `AGENTS.md` — cited, not restated.
|
|
45
46
|
- If building or maintaining a derived LLM wiki overlay (`${paths.wiki_dir}/`) -> read `agents/llm-wiki-methodology.md` and invoke `wiki-curator` / `/capture`.
|
|
46
47
|
- For day-to-day status -> read and update `${paths.references_dir}/project-status.md`.
|
|
@@ -81,6 +82,9 @@ When adding or updating agents, update REGISTRY.md — not this file.
|
|
|
81
82
|
- `agents/component-manifest-schema.md` — Component Manifest Schema: the machine-readable frontmatter HEAD on each `components/<Name>.md` (deterministic-from-TS + agentic fields), on-demand read contract, regeneration + transition-leniency rules (since v4.65.0, gated on `features.has_design_system`)
|
|
82
83
|
- `agents/return-contract-protocol.md` — Subagent return-message economy: COMPACT (bounded headline + `path:line` findings + disk pointer) vs FULL, persist-then-summarize — the input-side twin of `effort-protocol.md` (since v4.59.0)
|
|
83
84
|
- `agents/runtime-portability-protocol.md` — Runtime-mechanics binding: the abstract-operation ↔ Claude/Codex map (spawn / permissions / workflow-accel / state-spine / decision-gate / read-write path / adversarial-vs-cross-model), detect-once capability contract; cited by `/new` + `/prd`. The runtime-mechanics twin of `effort-protocol.md` + `return-contract-protocol.md` (since the Codex-parity S4 wave)
|
|
85
|
+
- `agents/agent-operating-protocol.md` — Shared operating procedures for the core agents (injection guard, doc-retrieval consumption, persistent-memory hygiene, tool-budget discipline), `SECTION=` dispatch, read only the matching section; agents keep 1-line binding versions inline (since v5.0.0)
|
|
86
|
+
- `agents/review-protocol.md` — The shared verification engine for reviewers: Challenge + Actionability, Simulation (diff-walk / plan-walk), Chain-of-Verification, quantified risk scoring + absolute severity calibration, specialist-spawn discipline incl. orchestrated-mode `specialist_dispatch` suppression; `SECTION=` dispatch (since v5.0.0)
|
|
87
|
+
- `agents/doc-audit-protocol.md` — doc-reviewer's audit-mode procedures (drift validator suite, topological generation, epistemic metadata, SCIP anchors, schema/registry drift, coverage gauges); read ONLY when invoked without card context; `SECTION=` dispatch (since v5.0.0)
|
|
84
88
|
|
|
85
89
|
## Where to Document (Decision Tree)
|
|
86
90
|
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
# Review Protocol — the shared verification engine for BALDART reviewers
|
|
2
|
+
|
|
3
|
+
**Purpose**: the anti-hallucination and calibration passes every reviewer runs
|
|
4
|
+
— Challenge, Actionability, Simulation, Chain-of-Verification, quantified risk
|
|
5
|
+
scoring, specialist-spawn discipline — used to be duplicated (and drifting)
|
|
6
|
+
across `code-reviewer` and `plan-auditor`, with partial copies in
|
|
7
|
+
`security-reviewer` / `api-perf-cost-auditor`. This module is their **single
|
|
8
|
+
SSOT** (v5.0.0). Each reviewer keeps a 1-line BINDING version of every pass
|
|
9
|
+
inline and cites the matching section here for the full procedure. On
|
|
10
|
+
divergence, **this module wins**.
|
|
11
|
+
|
|
12
|
+
**What stays in the agent bodies, never here**: the pooled findings YAML
|
|
13
|
+
schema, verdict-line formats, severity never-demote lists, per-domain
|
|
14
|
+
checklists, spawn MATRICES (which specialist for which signal), and report
|
|
15
|
+
caps — those are I/O contracts and domain knowledge, not procedure.
|
|
16
|
+
|
|
17
|
+
**Consumers**: `code-reviewer`, `plan-auditor`, `security-reviewer`,
|
|
18
|
+
`api-perf-cost-auditor` (body citations); `doc-reviewer` cites
|
|
19
|
+
`SECTION=challenge` for its findings hygiene.
|
|
20
|
+
|
|
21
|
+
## Contract
|
|
22
|
+
|
|
23
|
+
- **Dispatch**: agents cite `agents/review-protocol.md SECTION=<challenge|simulation|cove|risk-scoring|specialist-spawn>`.
|
|
24
|
+
Grep for `### SECTION: <name>` and Read ONLY that section.
|
|
25
|
+
- **Order of passes** (normative): generate findings → **challenge** →
|
|
26
|
+
**simulation** (already-generated `simulation_failure` hypotheses included) →
|
|
27
|
+
**cove** on every survivor → **risk-scoring** → report. Specialist spawns
|
|
28
|
+
happen early (parallel with your own review); their findings enter YOUR
|
|
29
|
+
challenge + CoVe passes like your own.
|
|
30
|
+
- **Degrade-safe**: skipping a pass means less rigor, never a malformed report
|
|
31
|
+
— the report shape is defined in the agent body.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
### SECTION: challenge
|
|
36
|
+
|
|
37
|
+
**Challenge Pass** (before reporting). For EACH HIGH and MEDIUM finding ask:
|
|
38
|
+
|
|
39
|
+
> "What is the strongest argument that this is a false positive?"
|
|
40
|
+
|
|
41
|
+
Consider:
|
|
42
|
+
- Is this already handled elsewhere in the codebase?
|
|
43
|
+
- Is this a project convention I'm unfamiliar with (check the agent memory
|
|
44
|
+
false-positive list)?
|
|
45
|
+
- Is the issue intentionally deferred to a later card per `notes` / card scope?
|
|
46
|
+
- Am I applying a generic best practice that doesn't fit this context?
|
|
47
|
+
|
|
48
|
+
**Suppress** the finding if the FP argument is convincing, and record it:
|
|
49
|
+
|
|
50
|
+
```markdown
|
|
51
|
+
<details>
|
|
52
|
+
<summary>Suppressed findings (N items — challenge pass)</summary>
|
|
53
|
+
- **Finding title** — FP argument: <why suppressed>
|
|
54
|
+
</details>
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
**Never-demote guard**: items on the agent's never-demote list (each reviewer
|
|
58
|
+
declares its own inline) are NEVER false positives — do not suppress them,
|
|
59
|
+
regardless of how convincing the FP argument sounds.
|
|
60
|
+
|
|
61
|
+
**Actionability Pass** (after the FP challenge). A finding that survives the
|
|
62
|
+
challenge can still need NO change — the code/plan is fine as-is and the only
|
|
63
|
+
honest "fix" is "acceptable as-is / verified safe". That is a **cleared
|
|
64
|
+
concern, not a finding**: a downstream fixer spawned on it would no-op. Either
|
|
65
|
+
omit it, or — when the pooled schema exposes `requires_action` — set
|
|
66
|
+
`requires_action: false` (recorded, never routed to a writer). Record cleared
|
|
67
|
+
concerns separately:
|
|
68
|
+
|
|
69
|
+
```markdown
|
|
70
|
+
<details>
|
|
71
|
+
<summary>Verified, no action (N items)</summary>
|
|
72
|
+
- **Title** — why it is safe as-is
|
|
73
|
+
</details>
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Guard: actionability applies to MEDIUM/LOW only — a BLOCKER/HIGH is never
|
|
77
|
+
"no action". And if a change IS needed but is out of your scope, that is NOT
|
|
78
|
+
no-action: report it as a real finding (it surfaces as residual).
|
|
79
|
+
|
|
80
|
+
### SECTION: simulation
|
|
81
|
+
|
|
82
|
+
Walk the artifact as if you were the runtime (diff) or the implementing
|
|
83
|
+
engineer (plan). Emit findings of type `simulation_failure` with the exact
|
|
84
|
+
breaking point and the broken invariant.
|
|
85
|
+
|
|
86
|
+
**Diff variant** (code review) — for each non-trivial changed
|
|
87
|
+
function/handler:
|
|
88
|
+
1. **Input boundary**: feed it the messiest realistic input (empty, null,
|
|
89
|
+
malformed JSON, oversized payload, malicious script, expired token,
|
|
90
|
+
concurrent request from the same user). Where does it break first?
|
|
91
|
+
2. **State machine consistency**: if the change mutates persistent state (DB,
|
|
92
|
+
session/local storage, cookies), trace the state at each branch — does any
|
|
93
|
+
branch leave inconsistent state?
|
|
94
|
+
3. **Reversibility**: if the function fails mid-execution, can the partial
|
|
95
|
+
side effects be rolled back? A transactional/atomic write is OK; a sequence
|
|
96
|
+
of independent writes without a transaction wrapper is NOT.
|
|
97
|
+
4. **Concurrent runs**: two parallel requests on the same resource — where do
|
|
98
|
+
they collide?
|
|
99
|
+
5. **Loop boundary**: any `for`/`map` over unbounded data (collection without
|
|
100
|
+
limit, user-provided array)? Where does it explode?
|
|
101
|
+
|
|
102
|
+
**Plan variant** (plan audit) — for each plan step:
|
|
103
|
+
1. **Preconditions**: are all prerequisites from prior steps actually
|
|
104
|
+
satisfied at this point?
|
|
105
|
+
2. **State machine consistency**: what is the shared state (record, env var,
|
|
106
|
+
feature flag) at this step — consistent with later steps' assumptions?
|
|
107
|
+
3. **Reversibility**: if step N fails, can steps 1..N-1 roll back cleanly? If
|
|
108
|
+
not → `irreversible_step_without_safety_net`.
|
|
109
|
+
4. **Concurrent runs**: two instances of this plan running simultaneously
|
|
110
|
+
(parallel cards, retry, multiple environments) — where do they collide?
|
|
111
|
+
5. **External dependency clock**: async propagation (index build, DNS, CDN
|
|
112
|
+
purge, deploy) — is wait time accounted for?
|
|
113
|
+
|
|
114
|
+
**Hypothesis discipline (BINDING)**: simulation is an LLM mental walk, NOT
|
|
115
|
+
execution — every `simulation_failure` is a *hypothesis* until grounded.
|
|
116
|
+
Each one MUST survive the CoVe pass below (grep/read the actual code path);
|
|
117
|
+
set `cove_verified: true` only when the breaking branch is confirmed against
|
|
118
|
+
real code. Drop any simulation claim you cannot ground — never ship an
|
|
119
|
+
unverified "the runtime will break here" as HIGH.
|
|
120
|
+
|
|
121
|
+
### SECTION: cove
|
|
122
|
+
|
|
123
|
+
**Chain-of-Verification** (for every surviving HIGH/MEDIUM finding, after
|
|
124
|
+
Challenge + Simulation). Generate 2–3 verification questions per finding and
|
|
125
|
+
EXECUTE them via grep/read — this forces grounding of every citation in actual
|
|
126
|
+
evidence.
|
|
127
|
+
|
|
128
|
+
Example — finding "auth wrapper missing on POST handler at `<route-file>:45`"
|
|
129
|
+
(`<auth-wrapper>` = the project's auth guard, resolve from
|
|
130
|
+
`${paths.high_risk_modules}`):
|
|
131
|
+
1. Does the file exist? → `test -f <route-file>`
|
|
132
|
+
2. Is there really no auth import/wrapper? → `rg -n "<auth-wrapper>" <route-file>`
|
|
133
|
+
3. Is the route actually public per docs? → `rg -l "<route-path>" ${paths.references_dir}/api/`
|
|
134
|
+
|
|
135
|
+
Drop findings whose verification fails (the finding itself was wrong) and
|
|
136
|
+
record them:
|
|
137
|
+
|
|
138
|
+
```markdown
|
|
139
|
+
### Hallucinated Findings Dropped (CoVe)
|
|
140
|
+
- **Finding title** — Verification: `<command>` → `<result>` → dropped because <reason>
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
A finding whose evidence (file/line/quote) cannot be re-located verbatim is
|
|
144
|
+
dropped, not "approximately relocated".
|
|
145
|
+
|
|
146
|
+
### SECTION: risk-scoring
|
|
147
|
+
|
|
148
|
+
**Quantified risk** (mandatory on every HIGH finding, and on risk-register
|
|
149
|
+
rows):
|
|
150
|
+
- **Impact** (1–5): 1 = cosmetic, 5 = data loss / security breach / production
|
|
151
|
+
outage.
|
|
152
|
+
- **Likelihood** (1–5): 1 = theoretical only, 5 = will hit on first run.
|
|
153
|
+
- **Priority** = Impact × Likelihood (1–25).
|
|
154
|
+
|
|
155
|
+
Thresholds:
|
|
156
|
+
- Priority ≥ 16 → automatic **BLOCKER**.
|
|
157
|
+
- Priority 9–15 → confirms **HIGH**.
|
|
158
|
+
- Priority < 9 → demote to MEDIUM — unless the item is on the agent's
|
|
159
|
+
never-demote list.
|
|
160
|
+
|
|
161
|
+
**Severity calibration (BINDING)**: severity is **ABSOLUTE, per finding,
|
|
162
|
+
anchored to its evidenced consequence** — never a relative ranking, never a
|
|
163
|
+
quota. Positional bands ("top 20% → HIGH") manufacture mandatory findings on
|
|
164
|
+
clean artifacts and demote real defects on large batches; both are calibration
|
|
165
|
+
failures. For each survivor ask: *"what happens if this ships / is implemented
|
|
166
|
+
exactly as written?"* A HIGH MUST cite its evidence (file/field/line + the
|
|
167
|
+
code path that proves it); no evidence citation → cap at MEDIUM. **Zero HIGH —
|
|
168
|
+
or zero findings at all — on a clean artifact is a legitimate, reportable
|
|
169
|
+
outcome.** Do not stretch severities to fill bands.
|
|
170
|
+
|
|
171
|
+
### SECTION: specialist-spawn
|
|
172
|
+
|
|
173
|
+
Procedure for the specialist auto-spawn matrices (each reviewer keeps its OWN
|
|
174
|
+
matrix — signal → agent — inline; this section is the shared discipline):
|
|
175
|
+
|
|
176
|
+
1. **Orchestrated-mode suppression (BINDING — check FIRST).** If your spawn
|
|
177
|
+
prompt or lean contract carries a `specialist_dispatch` block
|
|
178
|
+
(`{"suppress": [...], "owner": ...}`) — or the legacy
|
|
179
|
+
`skip_doc_reviewer: true` flag — you MUST NOT spawn any agent listed in
|
|
180
|
+
`suppress`. The orchestrator owns specialist routing on that run (it runs
|
|
181
|
+
those reviews itself in dedicated phases); a nested spawn would duplicate
|
|
182
|
+
the review, untracked. Record what you would have routed as a normal
|
|
183
|
+
finding tagged `dispatch_deferred: <agent>` and let the orchestrator route
|
|
184
|
+
it. **Standalone invocations (no dispatch block): the matrix applies
|
|
185
|
+
normally.**
|
|
186
|
+
2. **At most once per specialist.** If a coordinating team has independently
|
|
187
|
+
spawned (or will spawn) a matched specialist on the same artifact, consume
|
|
188
|
+
its findings instead of re-spawning.
|
|
189
|
+
3. **Parallel, single message** — spawn all matched specialists in one message
|
|
190
|
+
with multiple Task calls.
|
|
191
|
+
4. **Structured returns**: instruct each specialist to return its structured
|
|
192
|
+
(Mode A / pooled YAML) findings schema, never free markdown — that is what
|
|
193
|
+
makes the merge lossless.
|
|
194
|
+
5. **Merge with provenance**: fold their findings into YOUR findings output
|
|
195
|
+
with their `source: <agent>` preserved. Fill any schema field the
|
|
196
|
+
specialist did not populate from their evidence, or set `N/A` — never drop
|
|
197
|
+
a real finding because a field is missing.
|
|
198
|
+
6. **Specialist findings pass through YOUR Challenge + CoVe** like your own.
|
|
199
|
+
7. **Deduplicate ownership**: when a specialist owns a domain's depth (e.g.
|
|
200
|
+
security-reviewer on an auth diff), do not emit your own duplicate findings
|
|
201
|
+
for the same issues — cover only what the specialist did NOT raise. On a
|
|
202
|
+
severity conflict, the specialist's rating wins.
|
|
203
|
+
8. **No generic substitution**: if a named specialist is unavailable, do NOT
|
|
204
|
+
substitute `general-purpose` or any generic agent — surface
|
|
205
|
+
`CAPABILITY_UNAVAILABLE: <agent>` in your verdict context and cover that
|
|
206
|
+
domain with your own persona checklist.
|
|
207
|
+
9. Zero specialist signals → no spawn; declare it in the verdict context.
|
|
@@ -258,7 +258,7 @@ After update, these files exist locally and are authoritative:
|
|
|
258
258
|
| Numeric reference tables (type / contrast / spacing / density) | `.framework/framework/agents/design-system-protocol.md` § "Reference Tables (embed)" |
|
|
259
259
|
| Token cascade primitive → semantic → component | `.framework/framework/agents/design-system-protocol.md` § "Token Cascade" |
|
|
260
260
|
| Upgraded `ui-expert` (states taxonomy, red flags, gates) | `.framework/framework/.claude/agents/ui-expert.md` |
|
|
261
|
-
| Code-reviewer
|
|
261
|
+
| Code-reviewer Post-Intervention Coherence (merge gate) | `.framework/framework/.claude/agents/code-reviewer.md` § "Design System Compliance" rule 7 (Post-Intervention Coherence Check) |
|
|
262
262
|
| `ui-design` Step H (design-time gate) | `.framework/framework/.claude/skills/ui-design/SKILL.md` |
|
|
263
263
|
| Performance Gates (CWV 2026, LCP/INP/CLS) | `.framework/framework/.claude/skills/frontend-design/SKILL.md` § "Performance Gates" |
|
|
264
264
|
| Motion + reduced-motion + View Transitions | `.framework/framework/.claude/skills/motion-design/reference/accessibility-and-modern-apis.md` |
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
name: finding-mine
|
|
2
|
+
description: Monthly deep mining of pooled findings, QA reports, batch trackers and reviewer memories into classified improvement proposals — the anticipatory loop behind the weekly skill-improve.
|
|
3
|
+
since_version: 5.0.0
|
|
4
|
+
|
|
5
|
+
schedule:
|
|
6
|
+
cron: "0 3 1 * *" # 1st of the month, 03:00 UTC (an hour after a weekly skill-improve slot)
|
|
7
|
+
timezone: UTC
|
|
8
|
+
jitter_minutes: 0
|
|
9
|
+
cadence_label: monthly
|
|
10
|
+
|
|
11
|
+
agent: skill-improver
|
|
12
|
+
|
|
13
|
+
prompt: |
|
|
14
|
+
MODE: miner
|
|
15
|
+
|
|
16
|
+
Run the monthly deep mining pass per `skill-improver.md § MODE: miner`:
|
|
17
|
+
|
|
18
|
+
1. Read the past 30 days of evidence — routine reports under `docs/reports/`
|
|
19
|
+
(including the month's weekly `*-skill-improve.md` and their Upstream
|
|
20
|
+
Candidates), `qa/*.md`, the `/new` batch trackers
|
|
21
|
+
(`$(git rev-parse --git-common-dir)/baldart/run/batch-tracker-*.md`,
|
|
22
|
+
mining `## Fix Application Log` incl. the `model=` fix-pass A/B outcomes
|
|
23
|
+
and `## Lessons Learned`), and the reviewer agent memories
|
|
24
|
+
(`.claude/agent-memory/{code-reviewer,plan-auditor,qa-sentinel}/MEMORY.md`).
|
|
25
|
+
Absent sources are skipped with a one-line note.
|
|
26
|
+
2. Produce CLASSIFIED proposals (each with ≥2 independent occurrences cited):
|
|
27
|
+
`author-time-coder-rule` | `never-demote-code-reviewer` |
|
|
28
|
+
`deterministic-gate-candidate` | `card-schema-field-proposal` (proposal
|
|
29
|
+
ONLY — never applied) | `upstream-candidate` (consolidated + deduplicated
|
|
30
|
+
from the weeklies; a weekly overlay fix that KEPT recurring escalates here
|
|
31
|
+
with the failed-fix evidence).
|
|
32
|
+
3. Apply at most 3 overlay edits yourself (the strongest, evidence-backed) —
|
|
33
|
+
write surface identical to the weekly pass: `.baldart/overlays/` + `docs/`
|
|
34
|
+
only, commits prefixed `[FINDING-MINE]`. Everything else stays a proposal.
|
|
35
|
+
4. Emit `docs/reports/{{YYYYMMDD}}-finding-mine.md` — proposals grouped by
|
|
36
|
+
class with evidence citations, confidence, and effort estimate.
|
|
37
|
+
|
|
38
|
+
Graceful degradation: no recurring pattern in the month → write a short
|
|
39
|
+
"no proposals" report and exit cleanly. Never edit framework files directly
|
|
40
|
+
(the framework-edit-gate denies it — the overlay is the write surface).
|
|
41
|
+
|
|
42
|
+
output:
|
|
43
|
+
path: docs/reports/{{YYYYMMDD}}-finding-mine.md
|
|
44
|
+
commit:
|
|
45
|
+
enabled: true
|
|
46
|
+
prefix: "[FINDING-MINE]"
|
|
47
|
+
branch: main
|
|
48
|
+
|
|
49
|
+
required_artifacts:
|
|
50
|
+
- docs/reports/
|
|
51
|
+
- .claude/agents/skill-improver.md
|
|
52
|
+
|
|
53
|
+
backend_hints:
|
|
54
|
+
- claude-code-cloud
|
|
55
|
+
- github-actions
|
|
56
|
+
- cron
|
|
@@ -89,3 +89,14 @@ routines:
|
|
|
89
89
|
registry context, runs lint, and commits directly to the trunk (no PR).
|
|
90
90
|
Optional (only runs when features.has_i18n is true and target languages
|
|
91
91
|
are configured).
|
|
92
|
+
|
|
93
|
+
- name: finding-mine
|
|
94
|
+
file: finding-mine.routine.yml
|
|
95
|
+
cadence: monthly
|
|
96
|
+
since_version: 5.0.0
|
|
97
|
+
summary: |
|
|
98
|
+
Monthly deep mining pass (skill-improver MODE: miner) — 30 days of
|
|
99
|
+
pooled findings, QA reports, batch trackers and reviewer memories
|
|
100
|
+
distilled into classified proposals (author-time rules, never-demote
|
|
101
|
+
rows, deterministic-gate candidates, card-schema proposals, upstream
|
|
102
|
+
candidates). The anticipatory loop behind the weekly skill-improve.
|
package/package.json
CHANGED
|
@@ -1487,6 +1487,21 @@ async function configure(opts = {}) {
|
|
|
1487
1487
|
fs.writeFileSync(outPath, serialize(merged), 'utf8');
|
|
1488
1488
|
UI.success(`Wrote ${CONFIG_FILE}`);
|
|
1489
1489
|
|
|
1490
|
+
// v5.0.0 — slot-generated agents depend on the flags just written (features.*,
|
|
1491
|
+
// stack.database, …): re-run the agent merge so `.claude/agents/` reflects the
|
|
1492
|
+
// new config immediately (closes the "config changed without update" gap).
|
|
1493
|
+
// Fail-safe: configure must never die on a merge hiccup — doctor heals later.
|
|
1494
|
+
try {
|
|
1495
|
+
const SymlinkUtils = require('../utils/symlinks');
|
|
1496
|
+
const enabledTools = (merged.tools && Array.isArray(merged.tools.enabled) && merged.tools.enabled.length)
|
|
1497
|
+
? merged.tools.enabled : ['claude'];
|
|
1498
|
+
if (fs.existsSync(path.join(cwd, '.framework'))) {
|
|
1499
|
+
new SymlinkUtils(cwd).mergeAgents({ tools: enabledTools });
|
|
1500
|
+
}
|
|
1501
|
+
} catch (err) {
|
|
1502
|
+
UI.warning(`Agent re-merge after configure failed (${err.message}) — run \`npx baldart doctor\` to heal.`);
|
|
1503
|
+
}
|
|
1504
|
+
|
|
1490
1505
|
// Ensure overlays dir exists (user-owned)
|
|
1491
1506
|
const overlaysAbs = path.join(cwd, OVERLAYS_DIR);
|
|
1492
1507
|
if (!fs.existsSync(overlaysAbs)) {
|