do-better 1.0.0

package/do-better/references/verification.md

@@ -0,0 +1,129 @@
+# Reproduce-or-Kill — D2 Verification Protocol (F4 defense)
+
+Verification is separated from finding. The agent (or pass) that proposed a
+candidate never judges it. Every candidate is either **reproduced** — and
+becomes a verified finding with a reproduction record — or **killed** and
+counted. There is no third bucket: *unverified findings never reach output.*
+
+## Evidence-citation rules (apply everywhere, not just D2)
+
+1. **Canonical citation format**, inline in every artifact body:
+
+   ```
+   path/to/file.js:123@a1b2c3d
+   ```
+
+   `path` is repo-relative, `line` is 1-based, `@sha` is the 7–40 hex commit
+   sha the claim was read at. A claim is pinned to the code as it was, so
+   `refresh` can detect when the ground moved under it.
+
+2. **Deterministic verification, no LLM.** A citation verifies iff the file
+   exists in the worktree and `1 ≤ line ≤ line count`. This is a filesystem
+   check, never a model judgment.
+
+3. **Every finding carries ≥ 1 verified citation.** A claim line whose
+   citations all fail verification is removed and logged. Behavior-inventory
+   entries with zero verified citations are dropped with a warning.
+
+4. **Stale ≠ trusted.** When `refresh` finds a cited file changed since the
+   pinned sha, the claim is flagged stale, not silently believed (skill-rot
+   doctrine: a stale claim is misinformation with the voice of authority).
+
+## The protocol, per candidate
+
+### Step 1 — Citation check (deterministic)
+
+Verify the candidate's citation per the rules above. Fails → **killed**. No
+appeal: a finder that cited a nonexistent location fabricated evidence.
+
+### Step 2 — Mechanical reproduction (preferred)
+
+If the claim is mechanically checkable, ask the model to propose a `reproCmd`
+from a **whitelist of shapes** — nothing outside it is ever executed:
+
+- `node --test <file>` — run an existing test file.
+- `node -e "<snippet>"` — a self-contained check, snippet ≤ 500 chars.
+- A grep-equivalent pattern check, executed natively (no shell).
+
+Execution discipline: `cwd` = repo root, no shell (`shell:false`), 30-second
+timeout, stdout/stderr and exit code recorded verbatim. The command
+demonstrating the claim succeeds → **verified**, `method: "command"`, the full
+record stored in the finding.
+
+**Only a _demonstrative_ check may alone verify (H9).** A check is demonstrative
+when it genuinely ties to the claim: a repo-authored `node --test <file>`, or a
+grep with a **specific** pattern. A model-proposed `node -e` snippet and a
+catch-all grep (`.`, `.*`, empty, and the like — patterns that match virtually
+any non-empty file) are **not** demonstrative: they can exit 0 without showing
+anything, so a candidate backed only by one is not verified on the command — it
+falls through to the blind re-read (Step 3), which must independently CONFIRM.
+A non-demonstrative `node -e` is never executed at all, removing that
+prompt-injection surface for hostile brownfield repos.
+
+### Step 3 — Blind re-read (for non-runnable claims)
+
+Design and debt claims can't run. For these, a **fresh frontier context** —
+which has seen none of the finder's reasoning — receives only:
+
+- the bare claim (title + one-paragraph claim text), and
+- the cited code slice: the cited line ± 40 lines.
+
+It re-derives a verdict from the code alone:
+
+- **CONFIRM** — the code, read cold, supports the claim → verified,
+  `method: "reread"`, the verdict rationale recorded.
+- **KILL** — the code does not support the claim → killed.
+- **UNCERTAIN** — cannot confirm from the slice → **killed**. Uncertainty is
+  not a finding.
+
+The blindness is load-bearing (F2): a verifier shown the proposer's reasoning
+rubber-stamps it. Withhold everything but the claim and the code.
+
+### Step 4 — Fail closed
+
+- Unparseable verdict → killed.
+- Repro command errors, times out, or exits unexpectedly without demonstrating
+  the claim → killed.
+- Offline mode: only candidates whose reproduction is a deterministic native
+  check survive (`method: "static"`); everything requiring model judgment is
+  killed for the run and may be re-found online later.
+
+Killed candidates are **counted** (per dimension, in state) but never written
+as findings. The kill count is diagnostic, not shameful — a healthy finder
+over-generates and a healthy verifier prunes hard.
+
+## The reproduction record
+
+Every verified finding stores, in frontmatter and body:
+
+```
+reproduction:
+  method: command | reread | static
+  record: <the command + captured output, or the verdict rationale>
+  exitCode: <integer or null>
+```
+
+The record is what makes the finding **re-checkable**: `refresh` re-runs
+`command`/`static` reproductions and re-verdicts `reread` ones against the
+fresh slice. A finding that no longer reproduces is marked RESOLVED and its
+roadmap item flips to done — the living-document loop closes only because
+every finding ships with the means of its own re-verification.
+
+## The gate
+
+D2 passes its deterministic gate iff:
+
+1. every `(dimension × packet)` cell went dry within `MAX_PASSES` (see
+   [refute-charter.md](./refute-charter.md)) — the deep-read set is partitioned
+   into packets and each cell loops-until-dry independently, so the gate detail
+   names the dimension *and* the packet that stalled, and
+2. the count of unverified findings written is exactly **zero** — true by
+   construction, asserted anyway.
+
+An empty or unreadable deep-read set online is also a gate failure (exit 2):
+with no packets to examine, "dry" would be vacuously true, so starvation is
+surfaced rather than passed silently. Offline is exempt — it consumes no
+packets and runs a single deterministic static pass per dimension.
+
+Either condition failing is exit 2. The findings count is never a success
+metric (D11); ten verified findings beat a hundred plausible ones.

package/package.json

@@ -0,0 +1,20 @@
+{
+  "name": "do-better",
+  "version": "1.0.0",
+  "description": "Brownfield codebase analysis → verified findings → technical roadmap. The brownfield front-end to the ADLC.",
+  "type": "module",
+  "bin": { "do-better": "bin/cli.js", "dobetter": "bin/cli.js" },
+  "files": ["bin/", "src/", "do-better/"],
+  "engines": { "node": ">=18.0.0" },
+  "publishConfig": { "access": "public", "provenance": true },
+  "scripts": {
+    "test": "node --test",
+    "prepublishOnly": "node --test"
+  },
+  "keywords": ["brownfield", "codebase-analysis", "roadmap", "characterization-tests", "adlc"],
+  "author": "Chris Williams (@voodootikigod)",
+  "license": "MIT",
+  "repository": { "type": "git", "url": "git+https://github.com/voodootikigod/do-better.git" },
+  "bugs": { "url": "https://github.com/voodootikigod/do-better/issues" },
+  "homepage": "https://github.com/voodootikigod/do-better#readme"
+}

package/src/adlc.js

@@ -0,0 +1,331 @@
+// src/adlc.js — locate + spawn the five @adlc package CLIs and skill-mining
+// (SPEC §5 composition contracts). The aidlc bins import core via relative
+// paths, so they must run in place inside an aidlc checkout — never copied out.
+//
+// Absence is never an exception: every wrapper returns { skipped: true, reason }
+// when its tool is unavailable, and the CALLER applies (and declares) the
+// graceful degradation from the blueprint §5 table.
+
+import { existsSync } from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { OpError, log, makeExec } from "./utils.js";
+
+export const ADLC_TOOLS = ["parallax", "coldstart", "hollow-test", "behavior-diff", "preflight"];
+
+const DEFAULT_TIMEOUT_MS = 600000;
+const REASON_NOT_INSTALLED = "not installed";
+
+function defaultPackageRoot() {
+  // src/adlc.js → package root is one level up.
+  return path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
+}
+
+function isTruthyEnv(v) {
+  return v != null && v !== "" && v !== "0" && v !== "false";
+}
+
+function adlcBinPath(dir, name) {
+  return path.join(dir, "packages", name, "bin", `${name}.mjs`);
+}
+
+function probeAdlcDir(dir) {
+  const available = {};
+  for (const name of ADLC_TOOLS) {
+    available[name] = existsSync(adlcBinPath(dir, name));
+  }
+  return available;
+}
+
+function emptyAvailability() {
+  const available = {};
+  for (const name of ADLC_TOOLS) available[name] = false;
+  available["skill-mining"] = false;
+  return available;
+}
+
+// Locate an aidlc checkout (and a skill-mining checkout, separately).
+// Probe order (first hit wins): env.DOBETTER_ADLC_DIR → sibling of the
+// do-better package → sibling of the target repo → npx fallback. Setting
+// DOBETTER_NO_NPX disables the npx fallback (mode "absent") so air-gapped runs
+// and tests degrade deterministically instead of touching the network.
+export function locateAdlc({ env = process.env, packageRoot, targetRoot } = {}) {
+  const pkgRoot = packageRoot ?? defaultPackageRoot();
+  const tgtRoot = targetRoot ?? process.cwd();
+  const noNpx = isTruthyEnv(env.DOBETTER_NO_NPX);
+
+  let mode = "absent";
+  let dir = null;
+  const available = emptyAvailability();
+
+  const candidates = [];
+  if (env.DOBETTER_ADLC_DIR) {
+    candidates.push({ dir: path.resolve(env.DOBETTER_ADLC_DIR), explicit: true });
+  }
+  candidates.push({ dir: path.resolve(pkgRoot, "..", "aidlc") });
+  candidates.push({ dir: path.resolve(tgtRoot, "..", "aidlc") });
+
+  for (const candidate of candidates) {
+    const probed = probeAdlcDir(candidate.dir);
+    if (ADLC_TOOLS.some((name) => probed[name])) {
+      mode = "dir";
+      dir = candidate.dir;
+      for (const name of ADLC_TOOLS) available[name] = probed[name];
+      break;
+    }
+    if (candidate.explicit) {
+      log.warn(
+        `DOBETTER_ADLC_DIR=${candidate.dir} contains no @adlc tool bins ` +
+          `(expected packages/<name>/bin/<name>.mjs) — continuing probe.`
+      );
+    }
+  }
+
+  if (mode === "absent" && !noNpx) {
+    // npx fallback: each tool is verified lazily on first use; a failed npx
+    // resolve flips that tool to unavailable for the session.
+    mode = "npx";
+    for (const name of ADLC_TOOLS) available[name] = true;
+  }
+
+  // skill-mining is located separately (different repo layout: bin/cli.js).
+  let smMode = "absent";
+  let smDir = null;
+  const smCandidates = [];
+  if (env.DOBETTER_SKILL_MINING_DIR) {
+    smCandidates.push({ dir: path.resolve(env.DOBETTER_SKILL_MINING_DIR), explicit: true });
+  }
+  smCandidates.push({ dir: path.resolve(pkgRoot, "..", "skill-mining") });
+  smCandidates.push({ dir: path.resolve(tgtRoot, "..", "skill-mining") });
+  for (const candidate of smCandidates) {
+    if (existsSync(path.join(candidate.dir, "bin", "cli.js"))) {
+      smMode = "dir";
+      smDir = candidate.dir;
+      break;
+    }
+    if (candidate.explicit) {
+      log.warn(
+        `DOBETTER_SKILL_MINING_DIR=${candidate.dir} does not look like a skill-mining checkout ` +
+          `(no bin/cli.js) — continuing probe.`
+      );
+    }
+  }
+  if (smMode === "absent" && !noNpx) smMode = "npx";
+  available["skill-mining"] = smMode !== "absent";
+
+  return { mode, dir, available, skillMining: { mode: smMode, dir: smDir } };
+}
+
+// Heuristic: did the spawn fail because npx could not resolve the package
+// (vs the tool itself running and exiting non-zero)?
+function npxResolveFailed(result) {
+  if (result?.error?.code === "ENOENT") return true; // npx binary itself absent
+  if (result?.status === 0) return false;
+  const stderr = result?.stderr ?? "";
+  return (
+    /npm (?:ERR!|error)/i.test(stderr) &&
+    /E404|ETARGET|ENOTFOUND|could not determine executable/i.test(stderr)
+  );
+}
+
+function toToolResult(result, wantJson) {
+  const exitCode = Number.isInteger(result.status) ? result.status : null;
+  const stdout = result.stdout ?? "";
+  const stderr = result.stderr ?? "";
+  let json = null;
+  let parseFailed = false;
+  if (wantJson) {
+    try {
+      json = JSON.parse(stdout.trim());
+    } catch {
+      json = null;
+      parseFailed = true;
+    }
+  }
+  const ok = exitCode === 0;
+  const gateFailed = exitCode === 2;
+  // Universal adlc exit contract: 0 pass / 1 could-not-run / 2 gate failed.
+  // Anything that is neither pass nor gate-fail (incl. spawn failure / timeout
+  // → status null) is operational; an unparseable --json stdout is too.
+  const opError = (!ok && !gateFailed) || parseFailed;
+  return { skipped: false, ok, gateFailed, opError, exitCode, stdout, stderr, json };
+}
+
+function truncateForReason(text, max = 300) {
+  const t = String(text ?? "").trim();
+  return t.length > max ? `${t.slice(0, max)}…` : t;
+}
+
+// Spawn one adlc tool. Never throws for tool ABSENCE ({ skipped: true });
+// throws OpError only for caller bugs (unknown tool name, malformed args).
+// LLM key env vars pass through to the child via the inherited environment
+// (parallax/coldstart autodetect ANTHROPIC → OPENAI → GEMINI themselves).
+export function runAdlcTool(loc, name, args = [], { cwd, exec, timeoutMs = DEFAULT_TIMEOUT_MS } = {}) {
+  if (!ADLC_TOOLS.includes(name)) {
+    throw new OpError(`Unknown adlc tool "${name}" (expected one of: ${ADLC_TOOLS.join(", ")}).`);
+  }
+  if (!Array.isArray(args) || args.some((a) => typeof a !== "string")) {
+    throw new OpError(`adlc ${name}: args must be an array of strings.`);
+  }
+  if (!loc || loc.mode === "absent" || !loc.available?.[name]) {
+    return { skipped: true, reason: REASON_NOT_INSTALLED };
+  }
+  const run = exec ?? makeExec();
+  let result;
+  if (loc.mode === "dir") {
+    const binPath = adlcBinPath(loc.dir, name);
+    if (!existsSync(binPath)) {
+      loc.available[name] = false; // drift since locate — flip for the session
+      return { skipped: true, reason: REASON_NOT_INSTALLED };
+    }
+    result = run(process.execPath, [binPath, ...args], { cwd, timeout: timeoutMs });
+  } else if (loc.mode === "npx") {
+    result = run("npx", ["--yes", `@adlc/${name}`, ...args], { cwd, timeout: timeoutMs });
+    if (npxResolveFailed(result)) {
+      loc.available[name] = false; // session flip — sanctioned service-object mutation
+      return { skipped: true, reason: `npx could not resolve @adlc/${name}` };
+    }
+  } else {
+    return { skipped: true, reason: REASON_NOT_INSTALLED };
+  }
+  return toToolResult(result, args.includes("--json"));
+}
+
+// LLM-backed tools (parallax, coldstart) exit 1 when they cannot run at all —
+// most commonly no API key in env. Blueprint §5: treat as absent for that run
+// so callers apply the same declared degradation as for a missing tool.
+function couldNotRun(name, r) {
+  return {
+    skipped: true,
+    reason: `${name} could not run (exit 1): ${truncateForReason(r.stderr || r.stdout) || "no output"}`,
+  };
+}
+
+// D1 divergence gate. gate === true ⇔ exit 0 (divergence below threshold).
+export function runParallax(loc, { file, request, n = 3, threshold = 0.25, cwd, exec } = {}) {
+  if (!file && !request) {
+    throw new OpError("runParallax requires either { file } or { request }.");
+  }
+  const subject = file ? ["--file", file] : ["--request", request];
+  const args = [...subject, "--n", String(n), "--threshold", String(threshold), "--json"];
+  const r = runAdlcTool(loc, "parallax", args, { cwd, exec });
+  if (r.skipped) return r;
+  if (r.exitCode === 1) return couldNotRun("parallax", r);
+  const j = r.json ?? {};
+  return {
+    skipped: false,
+    gate: r.exitCode === 0,
+    score: typeof j.score === "number" ? j.score : null,
+    agreements: Array.isArray(j.agreements) ? j.agreements : [],
+    divergences: Array.isArray(j.divergences) ? j.divergences : [],
+    raw: r.json,
+  };
+}
+
+// D3 coldstart gate over .dobetter/backlog/tickets.json (byte-compatible with
+// aidlc's .adlc/tickets.json schema).
+export function runColdstart(loc, { ticketsPath, all = true, ticketId, cwd, exec } = {}) {
+  if (typeof ticketsPath !== "string" || ticketsPath.length === 0) {
+    throw new OpError("runColdstart requires { ticketsPath }.");
+  }
+  if (!all && (typeof ticketId !== "string" || ticketId.length === 0)) {
+    throw new OpError("runColdstart requires { ticketId } when all is false.");
+  }
+  const args = all
+    ? ["--all", "--tickets", ticketsPath, "--json"]
+    : [ticketId, "--tickets", ticketsPath, "--json"];
+  const r = runAdlcTool(loc, "coldstart", args, { cwd, exec });
+  if (r.skipped) return r;
+  if (r.exitCode === 1) return couldNotRun("coldstart", r);
+  const j = r.json ?? {};
+  return {
+    skipped: false,
+    ok: r.exitCode === 0,
+    results: Array.isArray(j.results) ? j.results : [],
+    raw: r.json,
+  };
+}
+
+// D4 hollow-test audit. cwd must be the TARGET repo.
+export function runHollowTest(loc, { testCmd, base, max = 20, cwd, exec } = {}) {
+  if (typeof testCmd !== "string" || testCmd.length === 0) {
+    throw new OpError("runHollowTest requires { testCmd }.");
+  }
+  if (typeof base !== "string" || base.length === 0) {
+    throw new OpError("runHollowTest requires { base }.");
+  }
+  const args = ["--test-cmd", testCmd, "--base", base, "--max", String(max), "--json"];
+  const r = runAdlcTool(loc, "hollow-test", args, { cwd, exec });
+  if (r.skipped) return r;
+  const j = r.json ?? {};
+  const summary =
+    j.summary && typeof j.summary === "object" ? j.summary : { total: 0, killed: 0, survived: 0 };
+  return {
+    skipped: false,
+    ok: r.ok,
+    gateFailed: r.gateFailed,
+    opError: r.opError,
+    summary,
+    mutants: Array.isArray(j.mutants) ? j.mutants : [],
+    raw: r.json,
+  };
+}
+
+// refresh regression detection. verb: "capture" | "compare"; args are the
+// verb's own argv (e.g. ["--config", cfg, "--out", out, "--json"]).
+export function runBehaviorDiff(loc, verb, args = [], { cwd, exec } = {}) {
+  if (verb !== "capture" && verb !== "compare") {
+    throw new OpError(`runBehaviorDiff verb must be "capture" or "compare", got "${verb}".`);
+  }
+  if (!Array.isArray(args) || args.some((a) => typeof a !== "string")) {
+    throw new OpError("runBehaviorDiff: args must be an array of strings.");
+  }
+  return runAdlcTool(loc, "behavior-diff", [verb, ...args], { cwd, exec });
+}
+
+// D4 env check. cwd must be the TARGET repo (preflight requires a git repo).
+export function runPreflight(loc, { testCmd, cwd, exec } = {}) {
+  const args = ["--json", ...(testCmd ? ["--test-cmd", testCmd] : [])];
+  const r = runAdlcTool(loc, "preflight", args, { cwd, exec });
+  if (r.skipped) return r;
+  const j = r.json ?? {};
+  return {
+    skipped: false,
+    ok: r.ok,
+    gateFailed: r.gateFailed,
+    opError: r.opError,
+    checks: Array.isArray(j.checks) ? j.checks : [],
+    verdict: typeof j.verdict === "string" ? j.verdict : r.ok ? "pass" : "fail",
+    failedNames: Array.isArray(j.failedNames) ? j.failedNames : [],
+    raw: r.json,
+  };
+}
+
+// D1 sub-step: mine latent skills from the target repo.
+export function runSkillMining(loc, { targetDir, offline = false, exec } = {}) {
+  if (typeof targetDir !== "string" || targetDir.length === 0) {
+    throw new OpError("runSkillMining requires { targetDir }.");
+  }
+  const sm = loc?.skillMining ?? { mode: "absent", dir: null };
+  if (!loc?.available?.["skill-mining"] || sm.mode === "absent") {
+    return { skipped: true, reason: REASON_NOT_INSTALLED };
+  }
+  const run = exec ?? makeExec();
+  const tail = ["mine", targetDir, ...(offline ? ["--offline"] : [])];
+  let result;
+  if (sm.mode === "dir") {
+    const binPath = path.join(sm.dir, "bin", "cli.js");
+    if (!existsSync(binPath)) {
+      loc.available["skill-mining"] = false;
+      return { skipped: true, reason: REASON_NOT_INSTALLED };
+    }
+    result = run(process.execPath, [binPath, ...tail], { timeout: DEFAULT_TIMEOUT_MS });
+  } else {
+    result = run("npx", ["--yes", "skill-mining", ...tail], { timeout: DEFAULT_TIMEOUT_MS });
+    if (npxResolveFailed(result)) {
+      loc.available["skill-mining"] = false;
+      return { skipped: true, reason: "npx could not resolve skill-mining" };
+    }
+  }
+  return { skipped: false, ok: result.status === 0, stdout: result.stdout ?? "" };
+}