do-better 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +265 -0
- package/bin/cli.js +262 -0
- package/do-better/SKILL.md +417 -0
- package/do-better/references/refute-charter.md +170 -0
- package/do-better/references/scoring.md +98 -0
- package/do-better/references/taxonomy.md +136 -0
- package/do-better/references/templates/charter-template.md +71 -0
- package/do-better/references/templates/finding-template.md +56 -0
- package/do-better/references/templates/rail-template.md +74 -0
- package/do-better/references/templates/roadmap-template.md +67 -0
- package/do-better/references/templates/ticket-template.md +78 -0
- package/do-better/references/verification.md +129 -0
- package/package.json +20 -0
- package/src/adlc.js +331 -0
- package/src/artifacts.js +580 -0
- package/src/charter.js +657 -0
- package/src/comprehend.js +553 -0
- package/src/identify.js +1119 -0
- package/src/llm.js +530 -0
- package/src/rail.js +533 -0
- package/src/refresh.js +256 -0
- package/src/roadmap.js +630 -0
- package/src/scan.js +313 -0
- package/src/state.js +260 -0
- package/src/utils.js +449 -0
|
@@ -0,0 +1,417 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: do-better
|
|
3
|
+
description: >-
|
|
4
|
+
Analyze an existing (brownfield) codebase and produce a defensible technical
|
|
5
|
+
roadmap: comprehension artifacts, verified findings with file:line evidence,
|
|
6
|
+
a prioritized Now/Next/Later roadmap, ADLC-ready backlog tickets, and
|
|
7
|
+
characterization rails. Use for "what should be better here", "audit this
|
|
8
|
+
codebase", "tech-debt roadmap", "modernization plan", "do better",
|
|
9
|
+
"brownfield analysis", or preparing legacy code for safe agentic execution.
|
|
10
|
+
license: MIT
|
|
11
|
+
user-invocable: true
|
|
12
|
+
keywords: [brownfield, codebase-analysis, roadmap, tech-debt, characterization-tests, findings, adlc, audit]
|
|
13
|
+
argument-hint: "[path] [--provider anthropic|gemini|openai|local] [--budget N] [--offline]"
|
|
14
|
+
metadata:
|
|
15
|
+
version: 0.1.0
|
|
16
|
+
author: Chris Williams (@voodootikigod)
|
|
17
|
+
homepage: https://github.com/voodootikigod/do-better
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
# Do Better
|
|
21
|
+
|
|
22
|
+
**do-better** is the practice of pointing an agent at an existing codebase and
|
|
23
|
+
asking: *what should be better here, in what order, and how do we prove we
|
|
24
|
+
didn't break anything?* The output is judgment + plan — not vibes, **verified
|
|
25
|
+
findings**: every claim carries file:line citations pinned to a commit SHA,
|
|
26
|
+
every finding survived reproduce-or-kill verification, and every roadmap item
|
|
27
|
+
hands off as a cold-start-tested ticket ready for execution.
|
|
28
|
+
|
|
29
|
+
do-better is the **brownfield front-end to the ADLC**. It is a sibling to
|
|
30
|
+
[skill-mining](https://github.com/voodootikigod/skill-mining) (extracts latent
|
|
31
|
+
*knowledge*) and [aidlc](https://github.com/voodootikigod/aidlc) (executes
|
|
32
|
+
builds): do-better decides *what* to do and *in what order*, and proves the
|
|
33
|
+
ground won't shift while you do it. It never executes fixes — that is ADLC's
|
|
34
|
+
job (D1 charter boundary).
|
|
35
|
+
|
|
36
|
+
This skill is **dual-mode**: any harness can run the workflow below manually,
|
|
37
|
+
section by section. The automated path is the CLI — `npx do-better run` — which
|
|
38
|
+
executes the same lifecycle with the same gates; the CLI loads this file and
|
|
39
|
+
the `references/` documents as its prompt sources, so the two modes cannot
|
|
40
|
+
drift. Prefer the CLI when available; use the manual workflow when you have a
|
|
41
|
+
harness and a repo but no CLI.
|
|
42
|
+
|
|
43
|
+
## When to use this skill
|
|
44
|
+
|
|
45
|
+
- "What should we improve in this codebase, and in what order?"
|
|
46
|
+
- "Audit this repo" / "find the tech debt" / "is this codebase healthy?"
|
|
47
|
+
- Inheriting or acquiring a codebase: due diligence with evidence.
|
|
48
|
+
- Before pointing execution agents (ADLC) at legacy code — produce the
|
|
49
|
+
tickets and the rails first.
|
|
50
|
+
- "Make me a modernization / stabilization / handoff plan."
|
|
51
|
+
- Re-run after work lands: `refresh` keeps the roadmap a living document.
|
|
52
|
+
|
|
53
|
+
Do NOT use it for greenfield builds (no code to comprehend) or to actually
|
|
54
|
+
apply fixes (D1: analysis + roadmap + rails only — handoff to ADLC P3/P4).
|
|
55
|
+
|
|
56
|
+
## What you produce (output contract)
|
|
57
|
+
|
|
58
|
+
Everything lands in a committed `.dobetter/` directory in the target repo:
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
.dobetter/
|
|
62
|
+
charter.md # D0 output, human-approved; dimension weights
|
|
63
|
+
comprehension/
|
|
64
|
+
codemap.md
|
|
65
|
+
architecture.md # intended design vs actual drift
|
|
66
|
+
behavior-inventory.md # KEYSTONE — observable behaviors: routes, jobs, CLIs, events
|
|
67
|
+
dependencies.md # versions, EOL/CVE, coupling hotspots
|
|
68
|
+
rails-map.md # covered vs load-bearing-but-untested
|
|
69
|
+
glossary.md # business terms ↔ code terms
|
|
70
|
+
coverage-manifest.md # deep-read X%, scanned Y% — declared, never silent
|
|
71
|
+
findings/ # one file per VERIFIED finding; file:line + commit-SHA evidence,
|
|
72
|
+
# dimension, severity, reproduction record
|
|
73
|
+
ROADMAP.md # executive deliverable (Now/Next/Later, declined, risk-of-inaction)
|
|
74
|
+
backlog/ # ADLC-shaped tickets + tickets.json (machine mirror)
|
|
75
|
+
rails/manifest.md # pointers — actual tests live in the repo's test tree
|
|
76
|
+
state.json # run history, SHA pins, per-phase spend, prior-roadmap hashes
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
All claims are pinned to commit SHA. Stale claims are **flagged, never
|
|
80
|
+
trusted** (skill-rot doctrine: a stale claim is misinformation with the voice
|
|
81
|
+
of authority). Mined skills (the 7th comprehension artifact) live where
|
|
82
|
+
skill-mining puts them (`.agents/skills/`) and ride along into execution.
|
|
83
|
+
|
|
84
|
+
## Options
|
|
85
|
+
|
|
86
|
+
CLI flags (manual mode: treat each as an instruction you honor by hand):
|
|
87
|
+
|
|
88
|
+
| Flag | Semantics | Fail-closed rule |
|
|
89
|
+
|---|---|---|
|
|
90
|
+
| `--provider anthropic\|gemini\|openai\|local` | Force LLM provider. Default: autodetect from env, Anthropic first (`ANTHROPIC_API_KEY` → `GEMINI_API_KEY` → `OPENAI_API_KEY`), then a configured local endpoint (`DOBETTER_LOCAL_BASE_URL` + `DOBETTER_LOCAL_MODEL`) last. | Named provider without its key/URL is an error, not a silent fallback. No key at all is an error naming the env vars and `--offline`. |
|
|
91
|
+
| `--budget <usd>` | Hard USD ceiling across all phases; per-phase spend recorded in `state.json`. | Projected overrun **refuses the call and stops** with resume instructions — completed work is preserved, never truncated silently. |
|
|
92
|
+
| `--offline` | No LLM calls: static analysis + structure-only artifacts. | Degradations are declared in `coverage-manifest.md` and gate records — never silent. Network/parse failures are NOT downgraded to offline; they fail. |
|
|
93
|
+
| `--model-cheap / --model-mid / --model-frontier <id>` | Override one tier's model (see Model tiering). | Model names are validated; shell-metacharacter names are rejected. |
|
|
94
|
+
| `--target <dir>` (or 2nd positional) | Target repo. Default `.`. | Must be a git repository — claims are SHA-pinned, so no git means no run. |
|
|
95
|
+
| `--approve` | (`charter`, `roadmap`) Approve the human-gated artifact as it now stands on disk. | Approval re-validates: charter must satisfy the taxonomy floor; roadmap requires a clean coldstart gate. |
|
|
96
|
+
| `--n <N>` / `--threshold <t>` | (`audit`) D1 parallax fan width (default 3) **and** the D2 finder-pool ceiling / divergence threshold (default 0.25). | Divergence ≥ threshold fails the gate (exit 2) — it is a finding about confusion, not noise to ignore. In D2, `--n` is the *maximum* pool width; the effective width is charter-weighted (table below), so a low-weight dimension never over-fans. Unset, the D2 pool ceiling is **1** (pooling is opt-in; lens rotation across passes is always on). |
|
|
97
|
+
| `--yes` | Skip confirmations (e.g. the D4 rails commit). | Never skips the two human gates — those have no bypass flag, by design. |
|
|
98
|
+
| `--json` | Machine summary on stdout. | — |
|
|
99
|
+
|
|
100
|
+
**D2 charter-weighted pool width** (fan `N` distinct-lens finders per pass, so
|
|
101
|
+
"dry" means the codebase is exhausted, not that one context converged). Given
|
|
102
|
+
`--n` as the ceiling:
|
|
103
|
+
|
|
104
|
+
| Charter weight | Effective pool width |
|
|
105
|
+
|---|---|
|
|
106
|
+
| 4–5 | `--n` (full width) |
|
|
107
|
+
| 2–3 | `max(1, floor(--n / 2))` |
|
|
108
|
+
| 1 | `1` (no pooling — identical to the single-finder pre-pool behavior) |
|
|
109
|
+
|
|
110
|
+
At `--n 1` every dimension is width 1 regardless of weight (`min(1, …) = 1`),
|
|
111
|
+
reproducing the pre-pool single-finder call counts exactly. Each pooled call
|
|
112
|
+
sees the refute charter plus exactly one lens
|
|
113
|
+
([references/refute-charter.md](references/refute-charter.md) `## Lenses`) —
|
|
114
|
+
never the catalog, never a sibling lens.
|
|
115
|
+
|
|
116
|
+
Exit codes (the adlc-universal contract): **0** success or a clean human-gate
|
|
117
|
+
pause with printed resume instructions · **1** operational error (bad input,
|
|
118
|
+
no provider, network, budget) · **2** deterministic gate failure (divergence,
|
|
119
|
+
unverified findings, coldstart gaps, rails red, hollow survivors).
|
|
120
|
+
|
|
121
|
+
## The lifecycle
|
|
122
|
+
|
|
123
|
+
```
|
|
124
|
+
D-1 Scan ──▶ D0 Charter ──▶ D1 Comprehend ──▶ D2 Identify ──▶ D3 Roadmap ──▶ D4 Rail ──▶ Handoff
|
|
125
|
+
(cheap (interview (7 artifacts, (refute + (score, seq, (pin (tickets →
|
|
126
|
+
facts) + weights) parallax) verify) tickets) behavior) ADLC P3/P4)
|
|
127
|
+
│ │ │ │ │
|
|
128
|
+
▼ ▼ ▼ ▼ ▼
|
|
129
|
+
[HUMAN GATE 1] divergence < dry (K=2) + coldstart clean rails green +
|
|
130
|
+
charter approved threshold + zero unverified + [HUMAN GATE 2] hollow audit
|
|
131
|
+
human skim roadmap approved
|
|
132
|
+
↻ refresh — idempotent living-document re-run
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
Exactly **two human gates** (D8): charter approval and roadmap approval. Every
|
|
136
|
+
other gate is deterministic or threshold-based. The pipeline is resumable
|
|
137
|
+
across multi-day pauses: `state.json` is the single source of truth for "what's
|
|
138
|
+
next", and every phase reads only `.dobetter/` artifacts — never another
|
|
139
|
+
phase's in-memory results.
|
|
140
|
+
|
|
141
|
+
CLI: `npx do-better run` executes the whole pipeline, stopping cleanly at each
|
|
142
|
+
human gate; re-invoking after `--approve` resumes where it left off.
|
|
143
|
+
Phase-per-command (`scan`, `charter`, `audit`, `roadmap`, `rail`, `refresh`)
|
|
144
|
+
runs any step alone.
|
|
145
|
+
|
|
146
|
+
## D-1 — Scan (cheap, fast, factual)
|
|
147
|
+
|
|
148
|
+
Goal: real code facts so the charter interview asks informed questions
|
|
149
|
+
("CI takes 40 minutes — pain point?") instead of generic ones.
|
|
150
|
+
|
|
151
|
+
1. Require a git repo; record HEAD SHA.
|
|
152
|
+
2. Collect deterministically (no model needed): file count and LOC, extension
|
|
153
|
+
histogram, biggest files and directories, the **incantations** (package
|
|
154
|
+
scripts, Makefile targets, CI workflows, Dockerfiles), manifests and
|
|
155
|
+
dependency counts, TODO/FIXME/HACK density, test directories, README
|
|
156
|
+
excerpt.
|
|
157
|
+
3. Draft `comprehension/codemap.md` (cheap tier): one-line purpose per
|
|
158
|
+
top-level directory and major file, marked `draft: true` — D1 verifies it.
|
|
159
|
+
Offline: structure-only tree, purposes marked `(structure-only)`.
|
|
160
|
+
|
|
161
|
+
CLI: `npx do-better scan`.
|
|
162
|
+
|
|
163
|
+
## D0 — Charter (the grill-me interview)
|
|
164
|
+
|
|
165
|
+
Goal: an approved engagement charter that weights the 8-dimension taxonomy
|
|
166
|
+
floor and captures pain, 12-month intent, and constraints. The taxonomy floor
|
|
167
|
+
([references/taxonomy.md](references/taxonomy.md)) is non-negotiable: all 8
|
|
168
|
+
dimensions appear with weight ≥ 1 — the floor is the defense against charter
|
|
169
|
+
blind spots.
|
|
170
|
+
|
|
171
|
+
1. Synthesize ≤12 interview questions (frontier tier), each citing a concrete
|
|
172
|
+
scan fact and carrying a recommended answer.
|
|
173
|
+
2. **Codebase-check clause**: before asking anything, answer what the scan
|
|
174
|
+
facts already answer (empty test dirs speak for themselves). Auto-answered
|
|
175
|
+
questions are recorded under `## Established from the codebase`, with
|
|
176
|
+
citations — never asked.
|
|
177
|
+
3. Interview the stakeholder strictly one question at a time; empty answer
|
|
178
|
+
accepts the recommendation.
|
|
179
|
+
4. Synthesize `charter.md` per
|
|
180
|
+
[references/templates/charter-template.md](references/templates/charter-template.md):
|
|
181
|
+
intent (stabilize/scale/extend/handoff), per-dimension weights 1–5 with
|
|
182
|
+
rationale, engagement-specific extra dimensions, constraints.
|
|
183
|
+
5. **HUMAN GATE 1**: the stakeholder reviews/edits the file and approves. No
|
|
184
|
+
approval, no analysis — every downstream judgment is weighted by this
|
|
185
|
+
document.
|
|
186
|
+
|
|
187
|
+
CLI: `npx do-better charter`, then `npx do-better charter --approve`.
|
|
188
|
+
|
|
189
|
+
## D1 — Comprehend (seven artifacts, parallax-verified)
|
|
190
|
+
|
|
191
|
+
Goal: understand before judging. The **behavior inventory is the keystone** —
|
|
192
|
+
it is the denominator for "retain existing functionality."
|
|
193
|
+
|
|
194
|
+
1. **Coverage plan first (declared sampling, never silent)**: rank files by
|
|
195
|
+
charter-weight relevance × size × centrality; deep-read the top set
|
|
196
|
+
(~40 files / ~150KB), scan the next ~200 (signatures only), skip the rest.
|
|
197
|
+
Write `coverage-manifest.md` with deep/scanned/skipped percentages, the file
|
|
198
|
+
lists, the rationale, and a `## Degradations` section.
|
|
199
|
+
2. Produce the seven artifacts (mid-tier readers over chunked packets):
|
|
200
|
+
codemap (verifying the D-1 draft), architecture (intended vs actual drift),
|
|
201
|
+
**behavior-inventory** (`B-NNN` entries: kind route/cli/job/event/db-write,
|
|
202
|
+
surface, entry citation, summary — one bullet per observable behavior, every
|
|
203
|
+
one cited), dependencies, rails-map (behaviors × existing tests), glossary,
|
|
204
|
+
and mined skills (run skill-mining as a sub-step; skipped → declared).
|
|
205
|
+
3. **Citation gate (deterministic)**: every claim's `path:line@sha` citation is
|
|
206
|
+
verified against the worktree; claim lines whose citations all fail are
|
|
207
|
+
removed and logged; uncited behavior entries are dropped with a warning.
|
|
208
|
+
4. **Divergence gate (parallax)**: N fresh-context readings (default 3) of the
|
|
209
|
+
charter + behavior inventory + architecture narrative. Divergence below
|
|
210
|
+
threshold (default 0.25) passes; residual divergences seed D2
|
|
211
|
+
confusion-findings. Over threshold fails (exit 2) — confusion is a finding,
|
|
212
|
+
not noise. Parallax unavailable → declared single-reading degradation +
|
|
213
|
+
mandatory human skim.
|
|
214
|
+
|
|
215
|
+
CLI: `npx do-better audit` (runs D1 then D2).
|
|
216
|
+
|
|
217
|
+
## D2 — Identify (refute, then reproduce-or-kill)
|
|
218
|
+
|
|
219
|
+
Goal: findings that survive hostile scrutiny. Two separated roles, never the
|
|
220
|
+
same context: **finders** propose, **verifiers** kill.
|
|
221
|
+
|
|
222
|
+
1. **Packetize the whole deep-read set.** The readable deep-read files are
|
|
223
|
+
partitioned into finder **packets** (`partitionSlices`) — every file lands
|
|
224
|
+
in exactly one packet, in order, and a file too large for one packet becomes
|
|
225
|
+
its own hard-truncated singleton. This replaces the old "rotate one shared
|
|
226
|
+
≤30 KB window" scheme, which could only ever show the finder the head of the
|
|
227
|
+
set (and, with an oversized head slice, nothing at all). For each dimension
|
|
228
|
+
(all 8 + charter extras, descending weight) and each packet, run
|
|
229
|
+
fresh-context finder passes under the refutation charter
|
|
230
|
+
([references/refute-charter.md](references/refute-charter.md)): chartered to
|
|
231
|
+
REFUTE acceptability, file:line on every claim, low-confidence included.
|
|
232
|
+
Each pass sees one packet's code plus prior passes' **conclusions only**
|
|
233
|
+
(titles + files pool-wide across the dimension's packets, never transcripts —
|
|
234
|
+
a finding from packet 1 is never re-proposed against packet 3).
|
|
235
|
+
2. **Loop each (dimension × packet) cell until dry**: a pass with zero new
|
|
236
|
+
candidates is dry; stop at K=2 consecutive dry passes; a cell not dry within
|
|
237
|
+
8 passes fails the gate (the failure detail names the dimension AND the
|
|
238
|
+
packet). Admission is **two-layered**: (a) a free hash filter on
|
|
239
|
+
dimension + file + normalized claim rejects verbatim repeats; (b) online
|
|
240
|
+
only, each hash-survivor then faces a **cheap-tier semantic check** —
|
|
241
|
+
one `dedupe` call comparing it against prior admitted entries (this run's
|
|
242
|
+
pool **and** prior verified findings, D6) that share the same dimension AND
|
|
243
|
+
file. The judge is given each entry's `file:line` and the candidate's, so a
|
|
244
|
+
different code location is treated as a **distinct instance** even when the
|
|
245
|
+
wording matches — only a genuine re-wording of the *same* location is
|
|
246
|
+
suppressed (it does
|
|
247
|
+
not join the pool, does not become a finding, and does not count as "new" for
|
|
248
|
+
the dry streak, but its hash key is recorded so it is not re-litigated). This
|
|
249
|
+
semantic check is the **one sanctioned FAIL-OPEN path** in D2: an
|
|
250
|
+
unparseable response, an out-of-range index, or a network/parse error admits
|
|
251
|
+
the candidate as if new. Every other failure in D2 fails closed; this one
|
|
252
|
+
inverts deliberately, because a false-new costs only one wasted verification
|
|
253
|
+
call (which kills genuine junk anyway) while a false-duplicate permanently
|
|
254
|
+
loses a finding nothing downstream can resurrect. Offline runs skip layer (b)
|
|
255
|
+
entirely — hash-only, unchanged. An empty/unreadable deep-read set online is
|
|
256
|
+
a gate failure too — starvation is never a silent zero-finding pass. Packets that reached K=2 are recorded per head sha, so a
|
|
257
|
+
same-sha resume (after a `--budget` stop) skips them with zero re-issued
|
|
258
|
+
finder calls; a sha change discards that state and re-examines everything.
|
|
259
|
+
After the loop, a **`## D2 finder coverage`** section is written idempotently
|
|
260
|
+
into `comprehension/coverage-manifest.md`: per dimension the files examined,
|
|
261
|
+
packet count, total passes, and truncated slices, with unreadable deep-read
|
|
262
|
+
files under `### Unexamined`. Cost scales as
|
|
263
|
+
dimensions × packets × passes × poolN; `--budget` is the hard ceiling and a
|
|
264
|
+
mid-loop stop preserves every finding verified so far (findings are written
|
|
265
|
+
per candidate, not batched).
|
|
266
|
+
3. **Verify every candidate** per
|
|
267
|
+
[references/verification.md](references/verification.md): deterministic
|
|
268
|
+
citation check, then mechanical reproduction (whitelisted command shapes,
|
|
269
|
+
30s timeout) or blind frontier re-read of ONLY the cited slice ± 40 lines —
|
|
270
|
+
the finder's reasoning is withheld. CONFIRM → write the finding
|
|
271
|
+
([references/templates/finding-template.md](references/templates/finding-template.md))
|
|
272
|
+
with its full reproduction record. KILL / UNCERTAIN / unparseable → killed,
|
|
273
|
+
counted, never written.
|
|
274
|
+
4. Gate (deterministic): every dimension dry AND zero unverified findings
|
|
275
|
+
written. The findings count is never celebrated — it is a vanity metric.
|
|
276
|
+
|
|
277
|
+
## D3 — Roadmap (score, sequence, ticket, coldstart)
|
|
278
|
+
|
|
279
|
+
Goal: the executive deliverable plus a machine-readable backlog — dual
|
|
280
|
+
artifact, one source of truth.
|
|
281
|
+
|
|
282
|
+
1. **Living-document reconciliation**: on re-runs, findings that no longer
|
|
283
|
+
reproduce flip their items to `✅ done`; resolved items whose finding
|
|
284
|
+
re-verified flip to `⚠ regressed`.
|
|
285
|
+
2. **Score** per [references/scoring.md](references/scoring.md): frontier
|
|
286
|
+
proposes impact/effort (t-shirt) + confidence + dependencies; code computes
|
|
287
|
+
`impact × confidence ÷ effort`, charter-weighted. Omitted findings get
|
|
288
|
+
conservative defaults — nothing silently dropped.
|
|
289
|
+
3. **Sequence deterministically**: topological on dependencies, rails-first
|
|
290
|
+
Phase 0, quick wins front-loaded, Now/Next/Later by score band. Declined
|
|
291
|
+
items (score < 0.3 or explicit reason) get their own section with
|
|
292
|
+
risk-of-inaction — listed, never hidden.
|
|
293
|
+
4. Write `ROADMAP.md`
|
|
294
|
+
([references/templates/roadmap-template.md](references/templates/roadmap-template.md))
|
|
295
|
+
and one ticket per Now/Next item
|
|
296
|
+
([references/templates/ticket-template.md](references/templates/ticket-template.md)),
|
|
297
|
+
plus `backlog/tickets.json` in the exact ADLC schema.
|
|
298
|
+
5. **Coldstart gate**: every ticket is cold-start tested (a fresh agent with
|
|
299
|
+
only the ticket must be able to execute it). Gaps → repair up to 2 rounds →
|
|
300
|
+
still gapped → demote to Later; any Now/Next ticket still gapped fails the
|
|
301
|
+
gate (exit 2).
|
|
302
|
+
6. **HUMAN GATE 2**: stakeholder reviews `ROADMAP.md` + `backlog/`, edits
|
|
303
|
+
freely, approves.
|
|
304
|
+
|
|
305
|
+
CLI: `npx do-better roadmap`, then `npx do-better roadmap --approve`.
|
|
306
|
+
|
|
307
|
+
## D4 — Rail (pin behavior before anyone changes it)
|
|
308
|
+
|
|
309
|
+
Goal: characterization rails for the behaviors Phase-Now/Next items touch —
|
|
310
|
+
roadmap-scoped, not exhaustive (D7).
|
|
311
|
+
|
|
312
|
+
1. **Preflight env check first.** Red does NOT fail the phase: "Make the
|
|
313
|
+
environment runnable" becomes a Phase 0 roadmap item + ticket itself, and
|
|
314
|
+
rails are scoped to whatever is runnable.
|
|
315
|
+
2. Map Now/Next ticket scopes onto the behavior inventory → rail targets;
|
|
316
|
+
annotate rail-coverage gaps in rails-map.
|
|
317
|
+
3. Author rails per
|
|
318
|
+
[references/templates/rail-template.md](references/templates/rail-template.md):
|
|
319
|
+
fresh context (behavior entry + boundary I/O only — never implementation
|
|
320
|
+
internals), boundary-level golden-master style, **bug-compatible pinning**
|
|
321
|
+
(assert what IS, annotate `possibly a bug: see F-XXX`), into
|
|
322
|
+
`test/dobetter-rails/`.
|
|
323
|
+
4. **Rails green gate**: every rail green against current code (2 fix rounds,
|
|
324
|
+
then delete + record the gap — never ship a red rail).
|
|
325
|
+
5. **Hollow-test audit**: mutate the rails' assertions; survivors are vacuous
|
|
326
|
+
rails — fix once or delete + gap. Tool absent → deletion spot-check
|
|
327
|
+
(comment the behavior's entry line; the rail must go red).
|
|
328
|
+
6. Write `rails/manifest.md` (rows + gaps) and append rail paths to every
|
|
329
|
+
ticket's `rails` array so ADLC rails-guard **freezes them mechanically**.
|
|
330
|
+
|
|
331
|
+
CLI: `npx do-better rail`. Then handoff: each ticket in `.dobetter/backlog/`
|
|
332
|
+
is ready for ADLC P3/P4 intake.
|
|
333
|
+
|
|
334
|
+
## Refresh (the living document)
|
|
335
|
+
|
|
336
|
+
Run any time after work lands: `npx do-better refresh`.
|
|
337
|
+
|
|
338
|
+
1. Diff the repo against the pinned SHA — changed + untracked files only
|
|
339
|
+
(cost-proportional to drift, not repo size).
|
|
340
|
+
2. **Flag stale claims** in every artifact and finding citing changed files —
|
|
341
|
+
flagged, never trusted.
|
|
342
|
+
3. Re-verify stale findings by re-running their reproduction records: no longer
|
|
343
|
+
reproduces → `RESOLVED`, roadmap item → `✅ done`; still reproduces →
|
|
344
|
+
re-pinned to the new SHA.
|
|
345
|
+
4. Behavior-diff regression detection where runnable: a retained behavior that
|
|
346
|
+
changed with no roadmap item claiming the change is `⚠ regressed` — that is
|
|
347
|
+
the one refresh outcome that gate-fails (exit 2).
|
|
348
|
+
|
|
349
|
+
## Composition contracts
|
|
350
|
+
|
|
351
|
+
- **skill-mining** — invoked during D1; mined skills are comprehension
|
|
352
|
+
artifacts and ride along for the execution phase.
|
|
353
|
+
- **aidlc** — backlog tickets conform to the ADLC P2 ticket shape (atomic,
|
|
354
|
+
fresh-agent executable, explicit contracts, coldstart-tested);
|
|
355
|
+
`backlog/tickets.json` is consumable by `coldstart --tickets` unmodified.
|
|
356
|
+
Rails conform to P3 doctrine (separate context, frozen, hollow-audited).
|
|
357
|
+
Reused packages: `parallax` (D1 divergence), `coldstart` (D3 gate),
|
|
358
|
+
`hollow-test` (D4 audit), `preflight` (D4 env check), `behavior-diff`
|
|
359
|
+
(refresh regression detection).
|
|
360
|
+
- **grill-me** — the D0 interview is grill-me with a codebase-check clause,
|
|
361
|
+
seeded by D-1 facts.
|
|
362
|
+
|
|
363
|
+
Every absent tool degrades **gracefully and loudly**: the fallback is weaker
|
|
364
|
+
and says so, in `coverage-manifest.md` and the gate records. A degradation
|
|
365
|
+
that doesn't announce itself is a lie about coverage.
|
|
366
|
+
|
|
367
|
+
## Model tiering (cost of detecting error, not prestige)
|
|
368
|
+
|
|
369
|
+
| Tier | Used for | Why safe |
|
|
370
|
+
|---|---|---|
|
|
371
|
+
| cheap | D-1 scan summaries, codemap draft, dependency inventory, coldstart-fallback probes | Mechanical; errors caught instantly and deterministically |
|
|
372
|
+
| mid | D1 readers, D2 finders, D4 rail drafting | Adversarial verification + hollow-test catch their errors |
|
|
373
|
+
| frontier | D0 charter synthesis, D2 finding verdicts, D3 scoring/sequencing judgment | Errors here sail through every gate undetected |
|
|
374
|
+
|
|
375
|
+
## Anti-patterns (do not do these)
|
|
376
|
+
|
|
377
|
+
- **Celebrating findings count.** The refused vanity metric (D11). A hundred
|
|
378
|
+
plausible findings are worth less than ten verified ones; success is ticket
|
|
379
|
+
survival rate and zero retained-functionality regressions.
|
|
380
|
+
- **Silent sampling.** Reading 10% of the repo and writing as if you read it
|
|
381
|
+
all. The coverage manifest exists so sampling is *declared*; a bounded pass
|
|
382
|
+
that reads as exhaustive is a lie.
|
|
383
|
+
- **Trusting stale claims.** A citation pinned to a SHA the repo has moved past
|
|
384
|
+
is misinformation with the voice of authority. Flag it, re-verify it, never
|
|
385
|
+
quote it as current.
|
|
386
|
+
- **Self-verified findings.** The context that proposed a finding confirming
|
|
387
|
+
it. Verification is blind and separate, always.
|
|
388
|
+
- **Roadmaps without declined items.** A roadmap that hides what it chose not
|
|
389
|
+
to do hides its judgment. Declined + risk-of-inaction is mandatory.
|
|
390
|
+
- **Rails that assert the desired behavior.** Rails pin what IS,
|
|
391
|
+
bug-compatibly. Red-on-arrival rails are wishes, not rails.
|
|
392
|
+
- **Fixing things.** do-better analyzes, plans, and pins. Execution is ADLC's
|
|
393
|
+
charter; the moment you patch product code you've left this skill.
|
|
394
|
+
|
|
395
|
+
## Verification checklist
|
|
396
|
+
|
|
397
|
+
Before declaring a run done:
|
|
398
|
+
|
|
399
|
+
- [ ] `charter.md` carries all 8 taxonomy dimensions with weight ≥ 1 and is
|
|
400
|
+
human-approved (gate 1).
|
|
401
|
+
- [ ] `coverage-manifest.md` declares deep/scanned/skipped percentages and
|
|
402
|
+
every degradation taken.
|
|
403
|
+
- [ ] Behavior inventory exists; every entry has a verified `path:line@sha`
|
|
404
|
+
citation.
|
|
405
|
+
- [ ] Divergence gate passed below threshold (or single-reading degradation is
|
|
406
|
+
declared + human skim done).
|
|
407
|
+
- [ ] Every dimension ran to dry (K=2); every finding file has status
|
|
408
|
+
`verified` with a reproduction record; killed counts recorded.
|
|
409
|
+
- [ ] `ROADMAP.md` has Now/Next/Later, Done/Regressed, and a Declined section
|
|
410
|
+
with risk-of-inaction lines.
|
|
411
|
+
- [ ] Every Now/Next ticket passed coldstart (or is demoted + flagged);
|
|
412
|
+
`backlog/tickets.json` validates against the ADLC schema.
|
|
413
|
+
- [ ] Roadmap is human-approved (gate 2).
|
|
414
|
+
- [ ] Rails are green, hollow-audited (or spot-checked, declared), listed in
|
|
415
|
+
`rails/manifest.md` with gaps, and frozen into ticket `rails` arrays.
|
|
416
|
+
- [ ] `state.json` pins every completed phase to a SHA and records per-phase
|
|
417
|
+
spend.
|
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
# Refute Charter — D2 Finder Doctrine (F2/E4 defense)
|
|
2
|
+
|
|
3
|
+
This is the system prompt and operating doctrine for every D2 finder. A finder
|
|
4
|
+
is an adversarial reader chartered on exactly one quality dimension. Its job is
|
|
5
|
+
**not** to assess, summarize, or balance. Its job is to refute.
|
|
6
|
+
|
|
7
|
+
## The charter (system prompt)
|
|
8
|
+
|
|
9
|
+
> You are chartered to REFUTE the claim that this codebase is acceptable on
|
|
10
|
+
> dimension **{dimension label}**. Find concrete evidence that it is not.
|
|
11
|
+
> Every claim requires file:line citations. Report everything, including
|
|
12
|
+
> low-confidence findings — verification happens downstream. You do not decide
|
|
13
|
+
> what is worth reporting; the verification stage does. A finding you withhold
|
|
14
|
+
> out of politeness or uncertainty is a finding lost.
|
|
15
|
+
|
|
16
|
+
Append the dimension's finder-charter paragraph from
|
|
17
|
+
[taxonomy.md](./taxonomy.md) as the brief: it defines what to hunt and what
|
|
18
|
+
counts as evidence on this dimension.
|
|
19
|
+
|
|
20
|
+
## Why refutation, not assessment (F2/E4)
|
|
21
|
+
|
|
22
|
+
A model asked "is this code okay?" agrees with itself. A model asked to assess
|
|
23
|
+
produces balanced prose that surfaces nothing actionable. The failure modes this
|
|
24
|
+
doctrine defends against:
|
|
25
|
+
|
|
26
|
+
- **F2 — self-agreement**: the same context that formed an impression confirms
|
|
27
|
+
it. Defense: every pass runs in a **fresh context**; the finder never sees its
|
|
28
|
+
own earlier reasoning.
|
|
29
|
+
- **E4 — politeness collapse**: models soften findings into suggestions.
|
|
30
|
+
Defense: the charter explicitly reverses the burden — the codebase is presumed
|
|
31
|
+
unacceptable on the dimension until the finder runs out of evidence.
|
|
32
|
+
|
|
33
|
+
## Operating rules
|
|
34
|
+
|
|
35
|
+
1. **Fresh context per pass.** Each pass starts clean. Prior passes contribute
|
|
36
|
+
**conclusions only** — the titles and files of already-found candidates,
|
|
37
|
+
never transcripts or reasoning (F3 partitioning). This prevents anchoring
|
|
38
|
+
and lets each pass hunt where the last one didn't.
|
|
39
|
+
2. **Rotated evidence.** Successive passes see different deep-read slices of the
|
|
40
|
+
coverage plan first, so the search front moves instead of re-reading the
|
|
41
|
+
same files.
|
|
42
|
+
3. **Cite or it didn't happen.** Every candidate carries `file` and `line`. A
|
|
43
|
+
claim without a citation is dropped before verification, not argued with.
|
|
44
|
+
4. **Low confidence is reportable.** Confidence is a field (0–1), not a filter.
|
|
45
|
+
The finder reports; the verifier kills. Separating finding from verification
|
|
46
|
+
is the whole design — a finder that self-censors defeats it.
|
|
47
|
+
5. **No fix proposals.** Finders identify; D3 plans. A finder that drifts into
|
|
48
|
+
solutioning is burning tokens outside its charter.
|
|
49
|
+
|
|
50
|
+
## Output contract
|
|
51
|
+
|
|
52
|
+
Each pass returns JSON, nothing else:
|
|
53
|
+
|
|
54
|
+
```json
|
|
55
|
+
{
|
|
56
|
+
"candidates": [
|
|
57
|
+
{
|
|
58
|
+
"title": "short imperative summary of what is wrong",
|
|
59
|
+
"claim": "one paragraph: what is wrong, why it matters on this dimension, what triggers it",
|
|
60
|
+
"file": "relative/path/from/repo/root.js",
|
|
61
|
+
"line": 42,
|
|
62
|
+
"severity": "critical|high|medium|low",
|
|
63
|
+
"confidence": 0.7
|
|
64
|
+
}
|
|
65
|
+
]
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
An empty pass is `{ "candidates": [] }` — a legitimate and expected result that
|
|
70
|
+
moves the loop toward dry.
|
|
71
|
+
|
|
72
|
+
Validation (fail closed — applied by the harness, not negotiated with the
|
|
73
|
+
finder): `file` must be a safe relative path inside the repo (no `..`, no
|
|
74
|
+
absolute paths), `line` an integer ≥ 1, `severity` one of the four values,
|
|
75
|
+
`confidence` a number in 0..1. Invalid candidates are dropped and logged.
|
|
76
|
+
|
|
77
|
+
## Loop-until-dry (F6)
|
|
78
|
+
|
|
79
|
+
- Constants: `K_DRY = 2`, `MAX_PASSES = 8`.
|
|
80
|
+
- A pass that produces **zero new candidates** (after dedupe on
|
|
81
|
+
dimension + file + normalized claim) increments the dry streak; any new
|
|
82
|
+
candidate resets it.
|
|
83
|
+
- The dimension is **dry** after `K_DRY` consecutive dry passes. One dry pass
|
|
84
|
+
is luck; two is evidence the seam is mined out.
|
|
85
|
+
- A dimension that hits `MAX_PASSES` without going dry has not converged — the
|
|
86
|
+
identify gate fails (exit 2). Do not pretend an unconverged search is
|
|
87
|
+
complete.
|
|
88
|
+
|
|
89
|
+
## What happens next
|
|
90
|
+
|
|
91
|
+
Every surviving candidate goes to adversarial verification —
|
|
92
|
+
[verification.md](./verification.md) — where it is reproduced or killed.
|
|
93
|
+
The finder's reasoning is **withheld** from the verifier (blind re-derivation);
|
|
94
|
+
only the bare claim and the cited code travel forward. Unverified candidates
|
|
95
|
+
never reach output.
|
|
96
|
+
|
|
97
|
+
## Lenses
|
|
98
|
+
|
|
99
|
+
One finder is one reader with one bias. A pool of finders that share a model,
|
|
100
|
+
a prompt, and a temperature is not a search — it is the same reader answering
|
|
101
|
+
the same question N times, and "dry" then means "this reader converged," not
|
|
102
|
+
"the codebase is exhausted." To make a pass genuinely fan out, each call in a
|
|
103
|
+
pool is assigned exactly **one** lens below and sees only that lens's
|
|
104
|
+
paragraph appended to the charter — never this heading, never the catalog,
|
|
105
|
+
never a sibling lens. The lens does not relax the refutation charter; it aims
|
|
106
|
+
it. A finder still refutes acceptability with file:line citations — the lens
|
|
107
|
+
only decides where it points its suspicion first.
|
|
108
|
+
|
|
109
|
+
### exploit-author
|
|
110
|
+
|
|
111
|
+
Read as someone who wants this code to do something it was never meant to do.
|
|
112
|
+
Trace every path from an untrusted boundary — a request body, a filename, an
|
|
113
|
+
environment variable, a row that another tenant could have written — to a
|
|
114
|
+
sink that acts on it: a shell, a query, a template, a deserializer, a file
|
|
115
|
+
path, an outbound request. Injection, SSRF, path traversal, auth checks that
|
|
116
|
+
are missing or that guard the wrong resource, secrets that live in the source
|
|
117
|
+
tree, tokens that never expire, comparisons that leak timing. Your evidence is
|
|
118
|
+
the exact line where attacker-controlled data reaches the dangerous call and
|
|
119
|
+
the input that turns it hostile. A "defense-in-depth" that isn't actually
|
|
120
|
+
reached still counts as a hole until you can cite the check that stops you.
|
|
121
|
+
|
|
122
|
+
### oncall-3am
|
|
123
|
+
|
|
124
|
+
Read as the engineer the pager just woke. This code is on fire in production
|
|
125
|
+
and you have no author to ask — only the logs, and whatever the code chose to
|
|
126
|
+
tell you. Hunt for what fails silently and what fails un-diagnosably: a caught
|
|
127
|
+
exception that logs nothing, a network call with no timeout, a retry that
|
|
128
|
+
hides the original error, a state machine with no breadcrumb for how it got
|
|
129
|
+
wedged, a fallback that masks the outage instead of surfacing it. Your
|
|
130
|
+
evidence is the line where a real failure produces no signal, or the wrong
|
|
131
|
+
signal, and the incident that line would prolong. If you cannot tell from the
|
|
132
|
+
code what "broken" would look like at 3am, that opacity is itself the finding.
|
|
133
|
+
|
|
134
|
+
### new-hire-reader
|
|
135
|
+
|
|
136
|
+
Read as someone opening this file on their first day, trusting the names and
|
|
137
|
+
comments to mean what they say. Hunt for everything that would build a wrong
|
|
138
|
+
mental model: a function named for what it no longer does, a comment that
|
|
139
|
+
contradicts the code beneath it, a parameter whose meaning flips with its
|
|
140
|
+
value, a magic constant with no origin, an implicit ordering or coupling that
|
|
141
|
+
nothing local reveals, an abstraction that leaks the moment you use it as
|
|
142
|
+
documented. Your evidence is the line that a careful, honest reader would
|
|
143
|
+
reasonably misunderstand, and the concrete mistake that misunderstanding
|
|
144
|
+
invites. Confusion that a newcomer would hit is a maintainability defect, not
|
|
145
|
+
a matter of taste.
|
|
146
|
+
|
|
147
|
+
### performance-profiler
|
|
148
|
+
|
|
149
|
+
Read with a flamegraph in mind and an adversary's input sizes. Hunt for the
|
|
150
|
+
cost that hides inside innocent-looking code: a database call inside a loop, a
|
|
151
|
+
quadratic scan dressed as a nested `for`, synchronous or blocking I/O on a hot
|
|
152
|
+
path, an unbounded collection that grows with untrusted input, a cache that is
|
|
153
|
+
never invalidated or never hit, an allocation or serialization repeated per
|
|
154
|
+
item that could be done once. Your evidence is the line whose cost is
|
|
155
|
+
super-linear in something a caller controls, plus the input scale that turns
|
|
156
|
+
it into a stall or an out-of-memory. "Fast enough on my laptop" is not a
|
|
157
|
+
refutation of a claim about production load.
|
|
158
|
+
|
|
159
|
+
### staff-skeptic
|
|
160
|
+
|
|
161
|
+
Read as the staff engineer in a design review who has watched systems rot.
|
|
162
|
+
Look past the local line to the decision it encodes and ask how it ages. Hunt
|
|
163
|
+
for eroding boundaries: a module reaching across a layer it should not know
|
|
164
|
+
about, an invariant that the code depends on but nothing enforces, two sources
|
|
165
|
+
of truth for one fact, error handling that varies by call site because no
|
|
166
|
+
contract pins it, a "temporary" shape that is now load-bearing. Your evidence
|
|
167
|
+
is the line where intent and implementation have diverged, or where a
|
|
168
|
+
cross-cutting rule is assumed but unprotected, and the plausible future change
|
|
169
|
+
that would break it. You are refuting the claim that this design is sound, not
|
|
170
|
+
that it compiles today.
|