@benjaminthomas/spec-driven-dev 1.10.0 → 1.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -46,6 +46,7 @@ Asking an AI agent to "just build it" tends to fail in predictable ways:
46
46
  | One agent working alone is slow | **`spec-implement`** runs several coder agents in parallel, each in its own copy of the repo |
47
47
  | "It compiles" gets treated as "it works" | **`spec-verify`** drives the real running app and checks every acceptance criterion, with evidence |
48
48
  | Problems slip through because nobody reviews the whole change | **`spec-review`** checks the finished branch against the spec, your conventions, and the code, and blocks shipping on real problems |
49
+ | The agent treats a file that broke five times like any other | **GitRisk** scores files from git history (churn, bug fixes, dependents, security fixes), so risky files get a deeper look, a higher tier and targeted tests. Switch it on or off any time |
49
50
  | A new session forgets what was decided last time | **Project memory**: a small `specs/CONTEXT.md`, plus a "Resume here" block and a log in every spec, kept short so reloading them is cheap |
50
51
  | Broken code gets committed, or a PR is pushed by accident | **`spec-checkpoint`** commits only once lint, types, tests, and build all pass. **`spec-ship`** asks before it pushes |
51
52
 
@@ -246,6 +247,54 @@ lines that point to files that no longer exist. If you install
246
247
  [graphify](https://github.com/Graphify-Labs/graphify), the skills use it for cheaper code
247
248
  navigation. It's optional (`context.graph: off` turns it off), and nothing is installed for you.
248
249
 
250
+ ### GitRisk: which files have a history of breaking
251
+
252
+ `spec-driven-dev risk` scores every tracked code file from git history alone, with no model and no
253
+ tokens, and writes the top files to `specs/context/risk.md`:
254
+
255
+ ```
256
+ Risk = 0.30 × Churn + 0.35 × Bug Density + 0.20 × Fan-out + 0.15 × Security (each factor 0–100)
257
+ ```
258
+
259
+ | Factor | Weight | What it measures |
260
+ |---|---|---|
261
+ | Churn | 30% | how often the file changes, against the repo's 90th percentile |
262
+ | Bug Density | 35% | share of its commits that were bug fixes (`fix:` type or fix-like wording) |
263
+ | Fan-out | 20% | how many other files import it, against the repo's 90th percentile |
264
+ | Security | 15% | share of its commits that were security fixes (security, CVE, XSS, injection…) |
265
+
266
+ Tiers: critical ≥ 75, high ≥ 50, medium ≥ 25. When `risk.enabled` is on, the skills use it:
267
+ - `spec-investigate` digs into high and critical files in the change surface (past fix commits,
268
+ dependents) and lists them as **Risk Hotspots**;
269
+ - `spec-create` raises those tasks one tier and requires a test aimed at each file's past failure
270
+ pattern;
271
+ - `spec-implement`'s reviewer and `spec-review` check those tests and read the risky files line
272
+ by line.
273
+
274
+ **Turning it on or off, no YAML needed:**
275
+ - **In your AI tool:** say "turn GitRisk off" or "turn GitRisk on". `spec-setup` changes just that
276
+ setting and tells you what it means.
277
+ - **In a terminal:** run `spec-driven-dev settings` and pick with the arrow keys. It also switches
278
+ run-through mode. For scripts, use `spec-driven-dev settings --risk=off`.
279
+ - **By hand:** set `risk.enabled: on` or `off` in `.spec-driven.yml`.
280
+
281
+ Off means every skill ignores risk completely.
282
+
283
+ `spec-driven-dev risk --files=a.ts,b.ts` prints scores for chosen files. The weights are a starting
284
+ hypothesis: change them under `risk.weights` in `.spec-driven.yml` once you've backtested them
285
+ against your own past bugs.
286
+
287
+ **Does it change what the agent does?** `spec-driven-dev risk-eval --tickets=tickets.md` runs each
288
+ ticket twice, in fresh clones with risk on and off, through investigate and create on the same
289
+ model. It then compares what the agent did:
290
+ - risky files read and named in the investigation;
291
+ - risky files in the plan, and whether tests were planned for them;
292
+ - task tiers;
293
+ - cost.
294
+
295
+ Results go to `experiments/` with the transcripts, and every run, including failures, is appended to
296
+ `experiments/LOG.md`. It needs Claude Code.
297
+
249
298
  ## How the workflow fits together
250
299
 
251
300
  ```
@@ -314,6 +363,9 @@ It starts from the project memory (`specs/CONTEXT.md` and only the area files th
314
363
  treats it as a lead rather than evidence, and notes where the code contradicts it. If graphify is
315
364
  installed, it uses it to map the code more cheaply.
316
365
 
366
+ With GitRisk on, it also looks harder at high-risk files in the change surface (their past fix
367
+ commits and dependents) and lists them as **Risk Hotspots**.
368
+
317
369
  **You get:** `specs/<feature>/investigation.md`, with every finding labelled **VERIFIED**,
318
370
  **INFERRED**, **ASSUMPTION**, or **UNKNOWN**, a short **Relevant Context** digest for the next
319
371
  stages, and requirements numbered R1, R2… so they can be traced through to verification and review.
@@ -335,6 +387,9 @@ to write**, acceptance criteria, the requirements it **covers**, and the few **p
335
387
  that apply to it. Every requirement must map to a task, and the spec isn't finished until it does.
336
388
  It also starts the spec's `log.md` and **Resume here** block.
337
389
 
390
+ With GitRisk on, a task that touches a high-risk file goes up one tier and must include a test
391
+ aimed at that file's past failures.
392
+
338
393
  **Then:** read it, edit anything you disagree with, and only then run `spec-implement`.
339
394
 
340
395
  ### 3. `spec-implement`: build it, batch by batch
@@ -380,6 +435,8 @@ nothing out of scope), the project's recorded conventions, and the code itself:
380
435
  security, error handling, tests, and hot-path performance. It checks each finding against the code
381
436
  before reporting it, and reads the diff rather than the whole repo.
382
437
 
438
+ With GitRisk on, high-risk files get a line-by-line read of their error paths and callers.
439
+
383
440
  **You get:** `specs/<feature>/review.md`, with findings ranked blocker, should-fix, nit, or question,
384
441
  each with `file:line` and a suggested fix. Blockers stop `spec-ship` and run-through. It then asks
385
442
  whether to fix them, and fixes go through `spec-implement`'s fix loop only after you say yes. It also
@@ -443,6 +500,8 @@ specs/
443
500
  └── review.md ← spec-review: ranked findings; open blockers stop spec-ship
444
501
  specs/CONTEXT.md ← spec-setup / spec-checkpoint: project memory index (≤60 lines)
445
502
  specs/context/<area>.md ← details per area, loaded only when needed (≤150 lines)
503
+ specs/context/risk.md ← spec-driven-dev risk: GitRisk scores (regenerate; don't edit)
504
+ experiments/ ← spec-driven-dev risk-eval: on/off reports, transcripts, LOG.md
446
505
  ```
447
506
 
448
507
  These are plain Markdown files, so you can review them in a PR, edit them, or hand them to a
@@ -475,6 +534,11 @@ Built into the skills, not left to the agent's judgment:
475
534
 
476
535
  ## FAQ
477
536
 
537
+ **How do I turn GitRisk off (or on)?**
538
+ Say "turn GitRisk off" in your AI tool, or run `spec-driven-dev settings` and pick from the menu.
539
+ Off means every skill ignores risk scores. Turning it on again offers to score the files if that
540
+ hasn't been done yet.
541
+
478
542
  **Do I need Jira?**
479
543
  No. Use GitHub Issues, Linear, a transcript, a document, or a sentence of plain text. Jira is just
480
544
  one of the options.
@@ -565,6 +629,10 @@ spec-driven-dev --yes # accept every detected defaul
565
629
  spec-driven-dev doctor # check .spec-driven.yml against the real repos
566
630
  spec-driven-dev status # list unfinished specs and each one's next step
567
631
  spec-driven-dev context --areas=billing # print the project memory (index, or chosen areas)
632
+ spec-driven-dev settings # switch GitRisk and run-through on/off from a menu
633
+ spec-driven-dev risk # GitRisk: score files from git history → specs/context/risk.md
634
+ spec-driven-dev risk-eval --tickets=tickets.md # compare agent behaviour with the risk signal on vs off
635
+ # (--runs=3 --arms=on --timeout=20 --model=sonnet)
568
636
  spec-driven-dev --help # show all of the above
569
637
  ```
570
638
 
@@ -658,16 +726,16 @@ skips that step rather than failing.
658
726
 
659
727
  | Skill | Stage | SKILL.md lines | Words | Supporting files |
660
728
  |---|---|---:|---:|---:|
661
- | `spec-setup` | 0 · configure | 142 | 1,334 | 3 |
662
- | `spec-investigate` | 1 · understand | 354 | 3,007 | 1 |
663
- | `spec-create` | 2 · plan | 183 | 1,646 | 6 |
729
+ | `spec-setup` | 0 · configure | 162 | 1,534 | 3 |
730
+ | `spec-investigate` | 1 · understand | 370 | 3,152 | 1 |
731
+ | `spec-create` | 2 · plan | 195 | 1,746 | 6 |
664
732
  | `spec-implement` | 3 · build | 322 | 2,683 | 3 |
665
733
  | `spec-verify` | 4 · prove | 163 | 1,557 | 1 |
666
- | `spec-review` | 5 · review | 131 | 1,209 | 1 |
667
- | `spec-checkpoint` | 6 · commit | 131 | 1,020 | 0 |
734
+ | `spec-review` | 5 · review | 135 | 1,270 | 1 |
735
+ | `spec-checkpoint` | 6 · commit | 133 | 1,046 | 0 |
668
736
  | `spec-ship` | 7 · PR | 230 | 1,388 | 0 |
669
737
  | `generative-seo` | standalone | 255 | 2,209 | 7 |
670
- | **Total** | | **1,911** | **16,053** | **22** |
738
+ | **Total** | | **1,965** | **16,585** | **22** |
671
739
 
672
740
  - **8** skills: **1** setup + **6** workflow stages + **1** standalone
673
741
  - **4** tools supported natively (Claude Code, Codex CLI, Cursor, Antigravity), plus an MCP fallback
@@ -688,7 +756,9 @@ plugin/skills/<name>/references/ templates and prompts the skill loads when need
688
756
  plugin/ the Claude Code plugin (manifest + skills only, no scripts or hooks)
689
757
  .agents/skills/ symlinks so Codex/Antigravity pick the skills up inside this repo
690
758
  bin/spec-driven-dev.cjs launcher (runs on any Node, explains if it's too old)
691
- bin/cli.mjs the `spec-driven-dev` CLI: installer, setup menus, doctor, status, context
759
+ bin/cli.mjs the `spec-driven-dev` CLI: installer, setup menus, doctor, status, context, risk
760
+ bin/risk.mjs GitRisk scoring (churn, bug density, fan-out, security)
761
+ bin/risk-eval.mjs risk on/off experiment harness
692
762
  package.json npm package (ships bin/ and plugin/skills/)
693
763
  mcp-server/ MCP fallback server
694
764
  CHANGELOG.md what changed in each version
@@ -717,7 +787,7 @@ plugin/evals/ `claude plugin eval` cases
717
787
  1. Increase `version` in both `package.json` and `plugin/.claude-plugin/plugin.json` (the check fails if
718
788
  they differ).
719
789
  2. Add a `CHANGELOG.md` entry, then commit and push.
720
- 3. Tag the release and push the tag: `git tag v1.10.0 && git push origin v1.10.0`.
790
+ 3. Tag the release and push the tag: `git tag v1.11.0 && git push origin v1.11.0`.
721
791
  4. Watch the **publish** run in the repo's Actions tab. When it's green, the new version is on
722
792
  npm. Claude Code plugin users get it from the marketplace once the commit is on `main`.
723
793
 
package/bin/cli.mjs CHANGED
@@ -7,15 +7,19 @@
7
7
  // spec-driven-dev doctor check .spec-driven.yml against the real repos (exit 1 on problems)
8
8
  // spec-driven-dev status list specs with unfinished work and each one's "Resume here" next step
9
9
  // spec-driven-dev context print specs/CONTEXT.md; --areas=billing,api prints only those area files
10
+ // spec-driven-dev risk GitRisk: score files from git history into specs/context/risk.md (--files=a,b --json --top=N --since=…)
11
+ // spec-driven-dev settings switch features on or off from a menu (GitRisk, run-through); or --risk=on|off --after-approval=…
12
+ // spec-driven-dev risk-eval U8 experiment: run tickets with risk on vs off and compare agent behaviour (--tickets=<file> --runs=N --model=… --arms=on,off --timeout=20)
10
13
  // flags: --yes --global --tools=claude,cursor --skills=spec-create,... --no-setup --dir=<path>
11
14
  // --layout=single|monorepo|multi-repo --repos=~/Projects/frontend,/srv/backend --workspace=<config dir>
12
- // --packages=apps/web,packages/ui --after-approval=stop|run-through --project=new|existing --version
15
+ // --packages=apps/web,packages/ui --after-approval=stop|run-through --project=new|existing --risk=on|off --version
13
16
  import fs from "node:fs";
14
17
  import os from "node:os";
15
18
  import path from "node:path";
16
19
  import { execFileSync } from "node:child_process";
17
20
  import prompts from "prompts"; // arrow-key menus that work in PowerShell, cmd, macOS and Linux terminals
18
21
  import { fileURLToPath } from "node:url";
22
+ import { scoreRepo, riskMarkdown, normWeights, weightProblems, setConfigValue } from "./risk.mjs";
19
23
 
20
24
  const ROOT = path.dirname(path.dirname(fileURLToPath(import.meta.url)));
21
25
  const SKILLS_DIR = path.join(ROOT, "plugin", "skills");
@@ -34,7 +38,7 @@ const args = process.argv.slice(2);
34
38
  const flag = (n) => args.includes(`--${n}`);
35
39
  const opt = (n) => args.find((a) => a.startsWith(`--${n}=`))?.split("=")[1];
36
40
  const command = args.find((a) => !a.startsWith("-")) || "install";
37
- const YES = flag("yes") || flag("y") || !process.stdin.isTTY || ["doctor", "status", "context"].includes(command);
41
+ const YES = flag("yes") || flag("y") || !process.stdin.isTTY || ["doctor", "status", "context", "risk", "risk-eval"].includes(command);
38
42
  const DIR = path.resolve(opt("dir") || process.cwd());
39
43
 
40
44
  if (flag("version") || flag("v")) {
@@ -43,22 +47,23 @@ if (flag("version") || flag("v")) {
43
47
  }
44
48
  // No terminal to answer in (e.g. `!` in Claude Code, an IDE task, a pipe) and nothing passed explicitly:
45
49
  // stop rather than write a config from guesses.
46
- const ANSWER_FLAGS = ["yes", "y", "layout", "repos", "packages", "tools", "workspace", "after-approval", "skills", "global", "no-setup", "project"];
47
- if (["install", "setup"].includes(command) && !process.stdin.isTTY && !args.some((a) => ANSWER_FLAGS.some((f) => a === `--${f}` || a.startsWith(`--${f}=`)))) {
48
- console.log(`✗ No interactive terminal, so setup can't ask its questions. Nothing was changed.
50
+ const ANSWER_FLAGS = ["risk", "yes", "y", "layout", "repos", "packages", "tools", "workspace", "after-approval", "skills", "global", "no-setup", "project"];
51
+ if (["install", "setup", "settings"].includes(command) && !process.stdin.isTTY && !args.some((a) => ANSWER_FLAGS.some((f) => a === `--${f}` || a.startsWith(`--${f}=`)))) {
52
+ console.log(`✗ No interactive terminal, so ${command === "settings" ? "the settings menu can't open" : "setup can't ask its questions"}. Nothing was changed.
49
53
 
50
54
  This happens when it runs through another program: a "!" command in Claude Code, an editor task,
51
55
  or piped input. Either:
52
56
  • run it yourself in a terminal window (Terminal, iTerm, or your editor's terminal panel):
53
- npx @benjaminthomas/spec-driven-dev@latest ${command === "setup" ? "setup" : ""}
57
+ npx @benjaminthomas/spec-driven-dev@latest ${command === "install" ? "" : command}
54
58
  • or give the answers as flags, for example:
55
- npx @benjaminthomas/spec-driven-dev@latest setup --layout=multi-repo --repos=~/Projects/frontend,~/Projects/backend
59
+ npx @benjaminthomas/spec-driven-dev@latest ${command === "settings" ? "settings --risk=off" : "setup --layout=multi-repo --repos=~/Projects/frontend,~/Projects/backend"}
56
60
  (add --yes to accept everything it detects)`);
57
61
  process.exit(1);
58
62
  }
59
63
 
60
64
  if (flag("help") || flag("h")) {
61
- console.log(fs.readFileSync(fileURLToPath(import.meta.url), "utf8").split("\n").slice(1, 11).join("\n").replace(/^\/\/ ?/gm, ""));
65
+ const lines = fs.readFileSync(fileURLToPath(import.meta.url), "utf8").split("\n").slice(1);
66
+ console.log(lines.slice(0, lines.findIndex((l) => !l.startsWith("//"))).join("\n").replace(/^\/\/ ?/gm, "")); // the whole header comment
62
67
  process.exit(0);
63
68
  }
64
69
 
@@ -241,6 +246,7 @@ function toYaml(c) {
241
246
  out += "effort: # effort / reasoning level per tier (Claude Code, Codex): low | medium | high | xhigh | max | default\n"
242
247
  + MODEL_KEYS.map((t) => ` ${t}: ${EFFORT_DEFAULTS[t]}\n`).join("");
243
248
  out += `context: # project memory: {specs_dir}/CONTEXT.md (index) + context/<area>.md\n project: ${c.project || "existing"} # existing | new\n graph: auto # auto = use graphify if installed | off\n`;
249
+ out += "risk: # GitRisk: historical risk per file from git (spec-driven-dev risk). Weights are a hypothesis to calibrate\n enabled: on # on | off (off = skills ignore risk; risk-eval compares the two)\n since: \"2 years ago\"\n weights:\n churn: 0.30\n bug_density: 0.35\n fan_out: 0.20\n security: 0.15\n";
244
250
  out += `pipeline:\n after_approval: ${c.afterApproval || "stop"} # stop | run-through (implement → verify → review → checkpoint; ship always asks)\n`;
245
251
  return out + `specs_dir: ${y(c.specsDir)}\n`;
246
252
  }
@@ -515,10 +521,20 @@ function parseYaml(text) {
515
521
  return root;
516
522
  }
517
523
 
524
+ // A multi-repo config applies only in its own folder and inside the repos it lists, so a repo that
525
+ // merely sits under that folder (e.g. ~/Projects/other-project) doesn't pick up the workspace's rules.
526
+ function configApplies(f, dir) {
527
+ let c;
528
+ try { c = parseYaml(fs.readFileSync(f, "utf8")); } catch { return true; }
529
+ if (c.layout !== "multi-repo" || !Array.isArray(c.repos)) return true;
530
+ const here = real(dir), base = path.dirname(f);
531
+ return here === real(base) || c.repos.some((r) => { const p = real(path.resolve(base, expandHome(String(r.path || "")))); return here === p || here.startsWith(p + path.sep); });
532
+ }
533
+
518
534
  function findConfig(dir) {
519
535
  for (let d = dir; ; d = path.dirname(d)) {
520
536
  const f = path.join(d, ".spec-driven.yml");
521
- if (fs.existsSync(f)) return f;
537
+ if (fs.existsSync(f) && configApplies(f, dir)) return f;
522
538
  if (d === path.dirname(d) || d === HOME) break;
523
539
  }
524
540
  // Not above us: a multi-repo workspace elsewhere may list this repo.
@@ -577,6 +593,11 @@ function doctor() {
577
593
  if (c.context?.graph && !["auto", "off"].includes(c.context.graph)) problems.push(`context.graph "${c.context.graph}" must be auto or off`);
578
594
  const repoRoots = c.layout === "multi-repo" ? (c.repos || []).map((r) => path.resolve(root, expandHome(r.path))) : [root];
579
595
  checkContext(specs, c.specs_dir || "specs", repoRoots, problems, notes);
596
+ if (c.risk) {
597
+ if (c.risk.enabled && !["on", "off"].includes(c.risk.enabled)) problems.push(`risk.enabled "${c.risk.enabled}" must be on or off`);
598
+ problems.push(...weightProblems(c.risk.weights));
599
+ if (c.risk.enabled !== "off" && !fs.existsSync(path.join(specs, "context", "risk.md"))) notes.push(`risk is on but ${c.specs_dir || "specs"}/context/risk.md doesn't exist — run: spec-driven-dev risk`);
600
+ }
580
601
  console.log(`Checked ${file}`);
581
602
  for (const n of notes) console.log(` • ${n}`);
582
603
  for (const p of problems) console.log(` ✗ ${p}`);
@@ -620,6 +641,103 @@ function context() {
620
641
  }
621
642
  }
622
643
 
644
+ function loadConfig() {
645
+ const file = findConfig(DIR);
646
+ return { file, root: file ? path.dirname(file) : gitTop(DIR), c: file ? parseYaml(fs.readFileSync(file, "utf8")) : {} };
647
+ }
648
+
649
+ function risk() {
650
+ const { root, c } = loadConfig();
651
+ const bad = weightProblems(c.risk?.weights);
652
+ if (bad.length) throw new Error(bad.join("; "));
653
+ const weights = normWeights(c.risk?.weights), since = opt("since") || c.risk?.since || "2 years ago";
654
+ const repoDirs = c.layout === "multi-repo" ? (c.repos || []).map((r) => ({ dir: path.resolve(root, expandHome(r.path)), name: r.path })) : [{ dir: root, name: path.basename(root) }];
655
+ const scored = repoDirs.filter((r) => isRepo(r.dir)).map((r) => ({ ...r, ...scoreRepo(r.dir, { since, weights, prefix: c.layout === "multi-repo" ? r.name : "" }) }));
656
+ if (!scored.length) throw new Error(`No git repo found at ${root}`);
657
+ const rows = scored.flatMap((s) => s.rows).sort((a, b) => b.score - a.score || b.commits - a.commits);
658
+ const wanted = splitList(opt("files"));
659
+ if (wanted.length) {
660
+ const norm = (f) => f.replace(/\\/g, "/").replace(/^\.\//, "");
661
+ const hits = wanted.map((f) => rows.find((r) => r.file === norm(f) || r.file.endsWith("/" + norm(f))) || { file: norm(f), score: null, tier: "no history" });
662
+ if (flag("json")) console.log(JSON.stringify(hits, null, 2));
663
+ else for (const h of hits) console.log(h.score === null ? `${h.file}: no history (new, untracked or not code)` : `${h.file}: ${h.score} ${h.tier} (churn ${h.factors.churn}, bug density ${h.factors.bug_density}, fan-out ${h.factors.fan_out}, security ${h.factors.security})`);
664
+ return;
665
+ }
666
+ if (flag("json")) { console.log(JSON.stringify({ weights, since, rows }, null, 2)); return; }
667
+ const specs = path.join(root, c.specs_dir || "specs");
668
+ fs.mkdirSync(path.join(specs, "context"), { recursive: true });
669
+ const out = path.join(specs, "context", "risk.md");
670
+ fs.writeFileSync(out, riskMarkdown({ rows, repos: scored, weights, since, top: Number(opt("top")) || 40 }));
671
+ const count = (t) => rows.filter((r) => r.tier === t).length;
672
+ console.log(`✓ Scored ${rows.length} files from ${scored.reduce((t, s) => t + s.commits, 0)} commits → ${path.relative(DIR, out) || out}`);
673
+ console.log(` ${count("critical")} critical · ${count("high")} high · ${count("medium")} medium. Top 5:`);
674
+ for (const r of rows.slice(0, 5)) console.log(` ${String(r.score).padStart(5)} ${r.tier.padEnd(8)} ${r.file}`);
675
+ if (c.risk?.enabled === "off") console.log(" Note: risk.enabled is off in .spec-driven.yml, so the skills won't use this until you turn it on.");
676
+ }
677
+
678
+ async function riskEvalCmd() {
679
+ const { file, root, c } = loadConfig();
680
+ const repo = gitTop(DIR);
681
+ if (c.layout === "multi-repo" && real(repo) === real(root)) throw new Error("risk-eval runs on one repo: run it inside one of the workspace's repos");
682
+ const ticketsFile = opt("tickets");
683
+ if (!ticketsFile) throw new Error("Pass --tickets=<file> with one ticket per line");
684
+ const tickets = fs.readFileSync(path.resolve(DIR, ticketsFile), "utf8").split("\n").map((l) => l.replace(/^\s*[-*]\s+/, "").trim()).filter((l) => l && !l.startsWith("#"));
685
+ const claude = opt("claude") || "claude";
686
+ if (!opt("claude") && !onPath("claude")) throw new Error("risk-eval drives Claude Code headlessly; install it first (https://code.claude.com)");
687
+ const { riskEval } = await import("./risk-eval.mjs");
688
+ const runs = Number(opt("runs")) || 1, stages = splitList(opt("stages") || "investigate,create");
689
+ console.log(`GitRisk U8 experiment: ${tickets.length} ticket(s) × ${runs} run(s) × 2 arms (risk on / off) on ${root}`);
690
+ // A workspace config's repo paths don't resolve inside a lone clone, so a multi-repo member gets a single-repo config.
691
+ const configText = c.layout === "multi-repo" ? "version: 1\nlayout: single\n" : file ? fs.readFileSync(file, "utf8") : "";
692
+ const arms = splitList(opt("arms") || "on,off");
693
+ if (!arms.length || arms.some((a) => !["on", "off"].includes(a))) throw new Error("--arms must be on, off, or on,off");
694
+ const { dir } = await riskEval({ repo, tickets, arms, timeoutMin: Number(opt("timeout")) || 20, runs, model: opt("model") || "sonnet", stages, claude, out: path.resolve(DIR, opt("out") || "experiments"),
695
+ pluginDir: path.join(ROOT, "plugin"), since: c.risk?.since || "2 years ago", weights: c.risk?.weights, configText });
696
+ console.log(`✓ Report: ${path.join(dir, "report.md")}`);
697
+ }
698
+
699
+ // Plain-language switches, so anyone can change them without editing YAML.
700
+ const SETTINGS = [
701
+ { block: "risk", key: "enabled", flag: "risk", name: "GitRisk", about: "extra care on files with a history of bugs",
702
+ get: (c) => c.risk?.enabled || "on", options: [
703
+ { value: "on", title: "On", description: "investigations, plans and reviews look harder at historically risky files" },
704
+ { value: "off", title: "Off", description: "the skills ignore risk completely" }] },
705
+ { block: "pipeline", key: "after_approval", flag: "after-approval", name: "After you approve a spec", about: "how far it goes on its own",
706
+ get: (c) => c.pipeline?.after_approval || "stop", options: [
707
+ { value: "stop", title: "Stop after each stage", description: "you start implement, verify, review and commit yourself" },
708
+ { value: "run-through", title: "Run straight through", description: "implement → verify → review → commit; stops at any failure, never pushes" }] },
709
+ ];
710
+
711
+ async function settings() {
712
+ const file = findConfig(DIR);
713
+ if (!file) throw new Error(`No .spec-driven.yml in ${DIR} or above. Run: spec-driven-dev setup`);
714
+ let text = fs.readFileSync(file, "utf8");
715
+ const label = (s, v) => s.options.find((o) => o.value === v)?.title || v;
716
+ const save = (s, v) => {
717
+ if (!s.options.some((o) => o.value === v)) throw new Error(`--${s.flag} must be ${s.options.map((o) => o.value).join(" or ")}`);
718
+ text = setConfigValue(text, s.block, s.key, v);
719
+ fs.writeFileSync(file, text);
720
+ console.log(`✓ ${s.name}: ${label(s, v)} (saved in ${file})`);
721
+ };
722
+ const given = SETTINGS.filter((s) => opt(s.flag));
723
+ if (given.length) given.forEach((s) => save(s, opt(s.flag)));
724
+ else for (;;) {
725
+ const c = parseYaml(text);
726
+ const pick = await askSelect("Settings: pick one to change", [
727
+ ...SETTINGS.map((s) => ({ title: `${s.name}: ${label(s, s.get(c))}`, description: s.about, value: s })),
728
+ { title: "Done", value: null }], null);
729
+ if (!pick) break;
730
+ const v = await askSelect(`${pick.name}`, pick.options, pick.get(c));
731
+ if (v !== pick.get(c)) save(pick, v);
732
+ }
733
+ const c = parseYaml(text);
734
+ if (c.risk?.enabled !== "off" && !fs.existsSync(path.join(path.dirname(file), c.specs_dir || "specs", "context", "risk.md"))) {
735
+ if (given.length || !(await askConfirm("GitRisk is on but files haven't been scored yet. Score them now? (uses git only, takes seconds)", true)))
736
+ console.log(" GitRisk is on, but nothing is scored yet. Run: spec-driven-dev risk");
737
+ else risk();
738
+ }
739
+ }
740
+
623
741
  // Silent unless something is unfinished, so it's cheap to run at the start of a session.
624
742
  function status() {
625
743
  const file = findConfig(DIR);
@@ -648,11 +766,14 @@ try {
648
766
  if (command === "doctor") doctor();
649
767
  else if (command === "status") status();
650
768
  else if (command === "context") context();
769
+ else if (command === "risk") risk();
770
+ else if (command === "settings") await settings();
771
+ else if (command === "risk-eval") await riskEvalCmd();
651
772
  else if (command === "install") {
652
773
  await install();
653
774
  if (!flag("no-setup") && (await askConfirm("Set up this project now?", true))) await setup();
654
775
  } else if (command === "setup") await setup();
655
- else throw new Error(`Unknown command "${command}" — use install (default), setup, doctor, status, or context`);
776
+ else throw new Error(`Unknown command "${command}" — use install (default), setup, settings, doctor, status, context, risk, or risk-eval`);
656
777
  if (["install", "setup"].includes(command)) console.log('\nNext: in your AI tool, say "investigate <ticket>" or "create a spec".');
657
778
  } catch (e) {
658
779
  console.error(`✗ ${e.message}`);
@@ -0,0 +1,152 @@
1
+ // GitRisk U8 experiment: does giving the agent the risk signal change what it does?
2
+ // Each ticket runs twice in fresh clones of the repo, risk ON and risk OFF, with the same model and
3
+ // prompt. We then measure behaviour from what the agent read and wrote, and keep a dated log.
4
+ import fs from "node:fs";
5
+ import os from "node:os";
6
+ import path from "node:path";
7
+ import { execFileSync, spawnSync } from "node:child_process";
8
+ import { scoreRepo, riskMarkdown, normWeights, setConfigValue } from "./risk.mjs";
9
+
10
+ const TIER_RANK = { light: 1, standard: 2, deep: 3 };
11
+ const sh = (cwd, cmd, args) => execFileSync(cmd, args, { cwd, stdio: ["ignore", "pipe", "pipe"], maxBuffer: 512 * 1024 * 1024 }).toString();
12
+
13
+ export const setRiskEnabled = (text, on) => setConfigValue(text, "risk", "enabled", on ? "on" : "off");
14
+
15
+ // Behaviour metrics from the agent's transcript (stream-json) and the spec it wrote.
16
+ export function measure({ transcript, clone, risky, tracked = [] }) {
17
+ const reads = new Set();
18
+ let cost = 0, turns = 0, inTok = 0, outTok = 0, error = "";
19
+ for (const line of transcript.split("\n")) {
20
+ let e;
21
+ try { e = JSON.parse(line); } catch { continue; }
22
+ if (e.type === "assistant") for (const c of e.message?.content || []) {
23
+ // A file counts as looked at when a Read, Grep or Bash call names it (agents often use grep/sed/cat, not Read).
24
+ if (c.type !== "tool_use" || !["Read", "Grep", "Bash", "Glob"].includes(c.name)) continue;
25
+ const text = JSON.stringify(c.input || {}).split(clone + "/").join("").split(clone.replace(/^\/private/, "") + "/").join("");
26
+ for (const f of tracked) if (text.includes(f)) reads.add(f);
27
+ }
28
+ if (e.type === "result") {
29
+ cost = e.total_cost_usd || 0; turns = e.num_turns || 0;
30
+ inTok = (e.usage?.input_tokens || 0) + (e.usage?.cache_read_input_tokens || 0) + (e.usage?.cache_creation_input_tokens || 0);
31
+ outTok = e.usage?.output_tokens || 0;
32
+ if (e.is_error) error = String(e.result || "error").slice(0, 200);
33
+ }
34
+ }
35
+ const specsDir = path.join(clone, "specs");
36
+ const feature = (fs.existsSync(specsDir) ? fs.readdirSync(specsDir) : []).find((d) => fs.existsSync(path.join(specsDir, d, "investigation.md")) || fs.existsSync(path.join(specsDir, d, "tasks")));
37
+ const read = (f) => { try { return fs.readFileSync(f, "utf8"); } catch { return ""; } };
38
+ const inv = feature ? read(path.join(specsDir, feature, "investigation.md")) : "";
39
+ const tasks = [];
40
+ const tdir = feature && path.join(specsDir, feature, "tasks");
41
+ if (tdir && fs.existsSync(tdir)) for (const t of fs.readdirSync(tdir).filter((f) => f.endsWith(".md"))) {
42
+ const text = read(path.join(tdir, t));
43
+ const section = (name) => (text.match(new RegExp(`^## ${name}\\s*\\n([\\s\\S]*?)(?=^## |$(?![\\s\\S]))`, "m")) || [])[1] || "";
44
+ const files = [...(section("Files to Create") + section("Files to Modify")).matchAll(/`([^`\s]+\.[\w]+)`/g)].map((m) => m[1]);
45
+ const tests = section("Tests").trim();
46
+ tasks.push({ name: t, files, tier: (text.match(/\*\*Tier:\*\*\s*(light|standard|deep)/i) || [])[1]?.toLowerCase(), hasTests: !!tests && !/^none\b/i.test(tests) });
47
+ }
48
+ const isRisky = (f) => risky.has(f);
49
+ const planFiles = new Set(tasks.flatMap((t) => t.files));
50
+ const riskyPlan = [...planFiles].filter(isRisky);
51
+ const riskyTasks = tasks.filter((t) => t.files.some(isRisky));
52
+ const avg = (xs) => (xs.length ? Math.round((xs.reduce((a, b) => a + b, 0) / xs.length) * 100) / 100 : null);
53
+ return {
54
+ feature: feature || null,
55
+ files_read: reads.size,
56
+ risky_files_read: [...reads].filter(isRisky).length,
57
+ investigation_risky_mentions: [...risky].filter((f) => inv.includes(f)).length,
58
+ plan_files: planFiles.size,
59
+ risky_plan_files: riskyPlan.length,
60
+ risky_files_with_tests: riskyPlan.filter((f) => tasks.some((t) => t.files.includes(f) && t.hasTests)).length,
61
+ tier_on_risky_tasks: avg(riskyTasks.map((t) => TIER_RANK[t.tier]).filter(Boolean)),
62
+ tier_all_tasks: avg(tasks.map((t) => TIER_RANK[t.tier]).filter(Boolean)),
63
+ tasks: tasks.length,
64
+ cost_usd: Math.round(cost * 10000) / 10000, turns, input_tokens: inTok, output_tokens: outTok, error,
65
+ };
66
+ }
67
+
68
+ const METRICS = [
69
+ ["risky_files_read", "Risky files looked at"], ["investigation_risky_mentions", "Risky files named in investigation"],
70
+ ["risky_plan_files", "Risky files in the plan"], ["risky_files_with_tests", "…of those, with tests planned"],
71
+ ["tier_on_risky_tasks", "Avg tier of tasks on risky files (1–3)"], ["files_read", "Repo files looked at (Read/Grep/Bash)"],
72
+ ["tasks", "Tasks"], ["cost_usd", "Cost (USD)"], ["output_tokens", "Output tokens"],
73
+ ];
74
+
75
+ function report({ meta, results }) {
76
+ const lines = [`# GitRisk U8 experiment — ${meta.started}`, "",
77
+ "Question: does feeding GitRisk's historical-risk signal into the spec-driven-dev workflow change what the AI agent does?", "",
78
+ `- Repo: \`${meta.repo}\` at ${meta.head}; risky set = ${meta.risky} files scored high or critical (weights ${JSON.stringify(meta.weights)})`,
79
+ `- Model: ${meta.model} · stages: ${meta.stages.join(" → ")} · runs per arm: ${meta.runs} · tickets: ${meta.tickets.length}`,
80
+ "- Arms: **on** = risk.enabled on + specs/context/risk.md present; **off** = risk.enabled off, no risk file. Fresh clone per run, same prompt.",
81
+ "", "## Results (mean per arm)", "", "| Ticket | Metric | Off | On | Δ (on − off) |", "|---|---|---|---|---|"];
82
+ const mean = (rs, k) => { const v = rs.map((r) => r.metrics?.[k]).filter((x) => typeof x === "number"); return v.length ? Math.round((v.reduce((a, b) => a + b, 0) / v.length) * 100) / 100 : null; };
83
+ meta.tickets.forEach((t, i) => {
84
+ const off = results.filter((r) => r.ticket === i && r.arm === "off" && !r.metrics?.error), on = results.filter((r) => r.ticket === i && r.arm === "on" && !r.metrics?.error);
85
+ for (const [k, label] of METRICS) {
86
+ const a = mean(off, k), b = mean(on, k);
87
+ lines.push(`| ${i + 1}. ${t.slice(0, 40).replace(/\|/g, "/")} | ${label} | ${a ?? "—"} | ${b ?? "—"} | ${a !== null && b !== null ? Math.round((b - a) * 100) / 100 : "—"} |`);
88
+ }
89
+ });
90
+ const failed = results.filter((r) => r.metrics?.error || r.exit !== 0);
91
+ lines.push("", "## Runs", "", "| Ticket | Arm | Run | Exit | Feature folder | Error |", "|---|---|---|---|---|---|",
92
+ ...results.map((r) => `| ${r.ticket + 1} | ${r.arm} | ${r.run} | ${r.exit} | ${r.metrics?.feature || "—"} | ${(r.metrics?.error || "").replace(/\|/g, "/")} |`));
93
+ lines.push("", "## Caveats", "",
94
+ `- ${meta.runs} run(s) per arm is a pilot, not a significance test. Agent behaviour varies run to run; increase --runs before drawing conclusions.`,
95
+ "- Metrics are proxies read from the transcript and the written spec (files the agent opened, files and tests it planned, task tiers). They don't measure whether the eventual code has fewer bugs.",
96
+ "- The risky set comes from the same repo history the ON arm sees, so a higher \"risky files read\" in ON is expected; the interesting signal is whether that changes the plan (tests, tiers) without raising cost much.",
97
+ `- Failed runs: ${failed.length}. They are excluded from the means but kept in results.json.`);
98
+ return lines.join("\n") + "\n";
99
+ }
100
+
101
+ export async function riskEval(opts) {
102
+ const { repo, tickets, runs = 1, arms = ["on", "off"], timeoutMin = 20, model = "sonnet", stages = ["investigate", "create"], claude = "claude", out, pluginDir, since, weights, configText = "" } = opts;
103
+ if (!tickets.length) throw new Error("No tickets: pass --tickets=<file> with one ticket per line");
104
+ const started = new Date().toISOString().replace(/\.\d+Z$/, "Z");
105
+ const dir = path.join(out, `risk-eval-${started.replace(/[:]/g, "-")}`);
106
+ fs.mkdirSync(dir, { recursive: true });
107
+ const w = normWeights(weights);
108
+ const scored = scoreRepo(repo, { since, weights: w });
109
+ scored.rows.sort((a, b) => b.score - a.score);
110
+ const risky = new Set(scored.rows.filter((r) => r.tier === "high" || r.tier === "critical").map((r) => r.file));
111
+ const riskMd = riskMarkdown({ rows: scored.rows, repos: [{ name: path.basename(repo), head: scored.head, commits: scored.commits }], weights: w, since, top: 60 });
112
+ fs.writeFileSync(path.join(dir, "risk.md"), riskMd);
113
+ const meta = { started, repo, head: scored.head, risky: risky.size, weights: w, model, stages, runs, tickets };
114
+ const stageText = stages.map((s) => `spec-${s}`).join(", then ");
115
+ const results = [];
116
+ for (let t = 0; t < tickets.length; t++) for (let r = 1; r <= runs; r++) for (const arm of ((t + r) % 2 ? ["off", "on"] : ["on", "off"]).filter((a) => arms.includes(a))) {
117
+ const clone = fs.mkdtempSync(path.join(os.tmpdir(), `risk-eval-${arm}-`));
118
+ sh(os.tmpdir(), "git", ["clone", "-q", "--no-hardlinks", repo, clone]);
119
+ for (const extra of ["specs/CONTEXT.md", "specs/context"]) {
120
+ const src = path.join(repo, extra);
121
+ if (fs.existsSync(src) && !fs.existsSync(path.join(clone, extra))) fs.cpSync ? fs.cpSync(src, path.join(clone, extra), { recursive: true }) : sh(repo, "cp", ["-R", src, path.join(clone, extra)]);
122
+ }
123
+ fs.rmSync(path.join(clone, "specs/context/risk.md"), { force: true });
124
+ fs.writeFileSync(path.join(clone, ".spec-driven.yml"), setRiskEnabled(configText || "version: 1\nlayout: single\n", arm === "on"));
125
+ if (arm === "on") { fs.mkdirSync(path.join(clone, "specs/context"), { recursive: true }); fs.writeFileSync(path.join(clone, "specs/context/risk.md"), riskMd); }
126
+ const prompt = `Use the ticket-to-pr plugin skills. This is a non-interactive run: don't ask questions, make reasonable assumptions and record them. Ticket (plain text, no tracker): ${tickets[t]}\nRun ${stageText}. Work on the current branch; don't create branches, commit, push, or implement anything.`;
127
+ const res = spawnSync(claude, ["-p", "--output-format", "stream-json", "--verbose", "--model", model, "--plugin-dir", pluginDir, "--permission-mode", "acceptEdits",
128
+ "--allowedTools", "Read Write Edit Glob Grep Skill Bash(git:*) Bash(ls:*) Bash(cat:*) Bash(head:*) Bash(tail:*) Bash(wc:*) Bash(find:*) Bash(grep:*) Bash(node:*) Bash(npx:*) Bash(spec-driven-dev:*)"],
129
+ { cwd: clone, input: prompt, encoding: "utf8", maxBuffer: 512 * 1024 * 1024, timeout: timeoutMin * 60 * 1000 });
130
+ const transcript = res.stdout || "";
131
+ const tag = `t${t + 1}-${arm}-r${r}`;
132
+ fs.writeFileSync(path.join(dir, `${tag}.jsonl`), transcript);
133
+ const tracked = sh(clone, "git", ["ls-files"]).split("\n").filter((f) => f.includes("/") || f.includes("."));
134
+ const metrics = measure({ transcript, clone, risky, tracked });
135
+ if (res.status !== 0 && !metrics.error) {
136
+ // Say where it stopped, so a stall (usage limit, API wait) is told apart from a skill problem.
137
+ const events = transcript.split("\n").filter(Boolean).length;
138
+ const lastTool = [...transcript.matchAll(/"type":"tool_use","id":"[^"]+","name":"(\w+)"/g)].pop()?.[1] || "none";
139
+ metrics.error = `${res.error?.code === "ETIMEDOUT" ? `timed out after ${timeoutMin} min` : `exit ${res.status}`} · ${events} events · last tool ${lastTool} ${(res.stderr || "").slice(0, 120)}`.trim();
140
+ }
141
+ if (metrics.feature) fs.cpSync ? fs.cpSync(path.join(clone, "specs", metrics.feature), path.join(dir, tag), { recursive: true }) : sh(clone, "cp", ["-R", path.join("specs", metrics.feature), path.join(dir, tag)]);
142
+ results.push({ ticket: t, arm, run: r, exit: res.status, metrics });
143
+ fs.rmSync(clone, { recursive: true, force: true });
144
+ console.log(` ${tag}: ${metrics.error ? `failed (${metrics.error.slice(0, 80)})` : `read ${metrics.risky_files_read} risky / ${metrics.files_read} files, ${metrics.risky_plan_files} risky in plan, $${metrics.cost_usd}`}`);
145
+ }
146
+ fs.writeFileSync(path.join(dir, "results.json"), JSON.stringify({ meta, results }, null, 2));
147
+ const md = report({ meta, results });
148
+ fs.writeFileSync(path.join(dir, "report.md"), md);
149
+ const failed = results.filter((x) => x.metrics.error).length;
150
+ fs.appendFileSync(path.join(out, "LOG.md"), `${fs.existsSync(path.join(out, "LOG.md")) ? "" : "# GitRisk experiment log\n\nAppend-only record of every run, including failures.\n\n"}- ${started} · ${tickets.length} ticket(s) × ${runs} run(s) × 2 arms on \`${path.basename(repo)}\` at ${scored.head}, ${model}, ${stages.join("+")} · ${results.length - failed} ok, ${failed} failed · [report](${path.basename(dir)}/report.md)\n`);
151
+ return { dir, report: md, results };
152
+ }
package/bin/risk.mjs ADDED
@@ -0,0 +1,137 @@
1
+ // GitRisk: per-file historical risk from git alone (no model, no tokens).
2
+ // Risk = 0.30·Churn + 0.35·BugDensity + 0.20·FanOut + 0.15·Security (each factor 0–100)
3
+ // The weights are a hypothesis to calibrate by backtest, so .spec-driven.yml `risk.weights` overrides them.
4
+ import fs from "node:fs";
5
+ import path from "node:path";
6
+ import { execFileSync } from "node:child_process";
7
+
8
+ export const DEFAULT_WEIGHTS = { churn: 0.3, bug_density: 0.35, fan_out: 0.2, security: 0.15 };
9
+ export const TIERS = [[75, "critical"], [50, "high"], [25, "medium"], [0, "low"]];
10
+ const MIN_COMMITS = 3; // shortcut: a file with 1 commit that was a fix would score 100% bug density; divide by at least 3. Replace with Bayesian smoothing if calibration shows it matters.
11
+
12
+ // Classification uses the conventional-commit type when the repo uses one, and message keywords otherwise.
13
+ const BUG = /^(fix|hotfix|bugfix|revert)(\(.+\))?!?:|\b(fix(e[sd]|ing)?|bug|hotfix|regression|broken|crash(es|ed)?|patch(ed)?|defect)\b/i;
14
+ const SECURITY = /\b(security|cve-\d+|xss|csrf|ssrf|sqli|injection|vuln(erability)?|exploit|sanitiz(e|ation)|escap(e|ing) (html|input|output)|auth(entication|orization)? (bypass|fix)|privilege|secret(s)? (leak|exposed)|leak(ed|s)? (token|key|secret|credential)|rate.?limit|helmet|csp)\b/i;
15
+ const SKIP = /(^|\/)(node_modules|dist|build|out|vendor|coverage|\.next|graphify-out|specs)\/|\.(md|mdx|txt|lock|log|png|jpe?g|gif|svg|ico|webp|pdf|zip|map|snap|csv)$|(^|\/)(package-lock\.json|yarn\.lock|pnpm-lock\.yaml|Cargo\.lock|go\.sum|poetry\.lock|composer\.lock)$/i;
16
+ const CODE = /\.(m?[jt]sx?|cjs|py|rb|go|rs|java|kt|php|cs|swift|scala|vue|svelte)$/i;
17
+
18
+ const git = (cwd, args) => {
19
+ try { return execFileSync("git", args, { cwd, stdio: ["ignore", "pipe", "ignore"], maxBuffer: 512 * 1024 * 1024 }).toString(); } catch { return ""; }
20
+ };
21
+ export const classify = (subject) => ({ bug: BUG.test(subject), security: SECURITY.test(subject) });
22
+ export const tierOf = (score) => TIERS.find(([min]) => score >= min)[1];
23
+
24
+ // Value at the 90th percentile; "expected" in min(100, actual / expected × 100), so one outlier can't flatten everyone else.
25
+ function p90(values) {
26
+ const v = values.filter((x) => x > 0).sort((a, b) => a - b);
27
+ return v.length ? v[Math.min(v.length - 1, Math.floor(v.length * 0.9))] : 1;
28
+ }
29
+
30
+ // Within-repo dependents: how many other files import this one. Relative JS/TS imports and Python module paths.
31
+ // shortcut: no tsconfig path aliases or cross-repo consumers (GitRisk open question U7); add when a repo needs them.
32
+ function fanOut(repo, files) {
33
+ const set = new Set(files), dependents = new Map(files.map((f) => [f, new Set()]));
34
+ const exts = ["", ".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs", "/index.ts", "/index.tsx", "/index.js", "/index.jsx", ".py", "/__init__.py"];
35
+ const pyIndex = new Map(files.filter((f) => f.endsWith(".py")).map((f) => [f.replace(/\/__init__\.py$|\.py$/, "").split("/").join("."), f]));
36
+ for (const f of files) {
37
+ if (!CODE.test(f)) continue;
38
+ let text = "";
39
+ try { text = fs.readFileSync(path.join(repo, f), "utf8"); } catch { continue; }
40
+ if (text.length > 2e6) continue;
41
+ const targets = new Set();
42
+ for (const m of text.matchAll(/(?:from\s+|require\(\s*|import\(\s*|import\s+)["'](\.{1,2}\/[^"']+)["']/g)) {
43
+ const base = path.posix.normalize(path.posix.join(path.posix.dirname(f), m[1]));
44
+ const hit = exts.map((e) => base + e).find((c) => set.has(c));
45
+ if (hit) targets.add(hit);
46
+ }
47
+ if (f.endsWith(".py")) for (const m of text.matchAll(/^\s*(?:from\s+([\w.]+)\s+import|import\s+([\w.]+))/gm)) {
48
+ const hit = pyIndex.get(m[1] || m[2]);
49
+ if (hit) targets.add(hit);
50
+ }
51
+ for (const t of targets) if (t !== f) dependents.get(t).add(f);
52
+ }
53
+ return new Map([...dependents].map(([f, s]) => [f, s.size]));
54
+ }
55
+
56
+ // Score every tracked code file in one repo. `prefix` labels files in multi-repo workspaces.
57
+ export function scoreRepo(repo, { since = "2 years ago", weights = DEFAULT_WEIGHTS, prefix = "" } = {}) {
58
+ const files = git(repo, ["ls-files"]).split("\n").filter((f) => f && !SKIP.test(f));
59
+ const tracked = new Set(files);
60
+ const stats = new Map(files.map((f) => [f, { commits: 0, bug: 0, security: 0 }]));
61
+ let commits = 0;
62
+ // \x1e starts each commit; the subject follows, then the changed file names.
63
+ for (const block of git(repo, ["log", "--no-merges", `--since=${since}`, "--name-only", "--format=%x1e%s"]).split("\x1e").slice(1)) {
64
+ const [subject, ...changed] = block.split("\n");
65
+ const c = classify(subject);
66
+ commits++;
67
+ for (const f of changed) {
68
+ const s = tracked.has(f) && stats.get(f);
69
+ if (!s) continue;
70
+ s.commits++;
71
+ if (c.bug) s.bug++;
72
+ if (c.security) s.security++;
73
+ }
74
+ }
75
+ const fan = fanOut(repo, files);
76
+ const expChurn = p90([...stats.values()].map((s) => s.commits)), expFan = p90([...fan.values()]);
77
+ const rows = files.map((f) => {
78
+ const s = stats.get(f), denom = Math.max(s.commits, MIN_COMMITS);
79
+ const factors = {
80
+ churn: Math.min(100, (s.commits / expChurn) * 100),
81
+ bug_density: s.commits ? (s.bug / denom) * 100 : 0,
82
+ fan_out: Math.min(100, ((fan.get(f) || 0) / expFan) * 100),
83
+ security: s.commits ? (s.security / denom) * 100 : 0,
84
+ };
85
+ const score = Object.keys(DEFAULT_WEIGHTS).reduce((t, k) => t + (Number(weights[k]) || 0) * factors[k], 0);
86
+ const r = (x) => Math.round(x * 10) / 10;
87
+ return { file: prefix ? `${prefix}/${f}` : f, score: r(score), tier: tierOf(score), commits: s.commits, bug_fixes: s.bug, security_fixes: s.security, dependents: fan.get(f) || 0,
88
+ factors: { churn: r(factors.churn), bug_density: r(factors.bug_density), fan_out: r(factors.fan_out), security: r(factors.security) } };
89
+ });
90
+ return { commits, head: git(repo, ["rev-parse", "--short", "HEAD"]).trim(), rows };
91
+ }
92
+
93
+ export function normWeights(w) {
94
+ const out = { ...DEFAULT_WEIGHTS };
95
+ for (const k of Object.keys(DEFAULT_WEIGHTS)) if (w && w[k] !== undefined && w[k] !== "") out[k] = Number(w[k]);
96
+ return out;
97
+ }
98
+ export function weightProblems(w) {
99
+ const p = [];
100
+ for (const [k, v] of Object.entries(w || {})) {
101
+ if (!(k in DEFAULT_WEIGHTS)) p.push(`risk.weights.${k}: unknown factor (use ${Object.keys(DEFAULT_WEIGHTS).join(", ")})`);
102
+ else if (!(Number(v) >= 0)) p.push(`risk.weights.${k} "${v}" must be a number ≥ 0`);
103
+ }
104
+ const sum = Object.values(normWeights(w)).reduce((a, b) => a + b, 0);
105
+ if (Math.abs(sum - 1) > 0.001) p.push(`risk.weights add up to ${Math.round(sum * 1000) / 1000}, not 1`);
106
+ return p;
107
+ }
108
+
109
+ // The area file skills read: top N files, within the 150-line area-file limit.
110
+ export function riskMarkdown({ rows, repos, weights, since, top }) {
111
+ const pct = (x) => `${Math.round(x * 100)}%`;
112
+ const w = weights;
113
+ const shown = rows.filter((r) => r.score > 0).slice(0, Math.min(top, 120));
114
+ const lines = [
115
+ "# Risk",
116
+ "<!-- Generated by `spec-driven-dev risk`. Don't edit by hand: re-run it. Skills read this only when risk.enabled is on. -->",
117
+ `Scored ${repos.map((r) => `${r.name} at ${r.head} (${r.commits} commits)`).join(", ")} · since ${since} · ${new Date().toISOString().slice(0, 10)}`,
118
+ "",
119
+ `Risk = ${w.churn} × Churn + ${w.bug_density} × Bug Density + ${w.fan_out} × Fan-out + ${w.security} × Security (each 0–100; weights ${pct(w.churn)} / ${pct(w.bug_density)} / ${pct(w.fan_out)} / ${pct(w.security)}).`,
120
+ "Tiers: critical ≥ 75 · high ≥ 50 · medium ≥ 25. A historical signal, not proof of a bug.",
121
+ "",
122
+ "| File | Score | Tier | Churn | Bug density | Fan-out | Security | Commits (fixes / security) |",
123
+ "|---|---|---|---|---|---|---|---|",
124
+ ...shown.map((r) => `| \`${r.file}\` | ${r.score} | ${r.tier} | ${r.factors.churn} | ${r.factors.bug_density} | ${r.factors.fan_out} | ${r.factors.security} | ${r.commits} (${r.bug_fixes} / ${r.security_fixes}) |`),
125
+ ];
126
+ if (rows.length > shown.length) lines.push("", `${rows.length - shown.length} more files not shown. \`spec-driven-dev risk --files=a,b\` prints any file's score.`);
127
+ return lines.join("\n") + "\n";
128
+ }
129
+
130
+ // Change one `block: key: value` line in .spec-driven.yml text, keeping every comment and other line as is.
131
+ export function setConfigValue(text, blockName, key, value) {
132
+ const block = text.match(new RegExp(`^${blockName}:.*\\n(?:(?:[ \\t]+.*|[ \\t]*)\\n)*`, "m"));
133
+ const line = new RegExp(`^([ \\t]+${key}:[ \\t]*)[^\\s#]+`, "m");
134
+ if (!block) return `${text.replace(/\n*$/, "\n")}${blockName}:\n ${key}: ${value}\n`;
135
+ const body = line.test(block[0]) ? block[0].replace(line, `$1${value}`) : block[0].replace(new RegExp(`^${blockName}:.*\\n`), `$& ${key}: ${value}\n`);
136
+ return text.replace(block[0], body);
137
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@benjaminthomas/spec-driven-dev",
3
- "version": "1.10.0",
3
+ "version": "1.11.0",
4
4
  "description": "Spec-driven AI development workflow (investigate → spec → implement → verify → review → ship) as Agent Skills for Claude Code, Cursor, Codex CLI and Antigravity.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -8,7 +8,8 @@
8
8
  },
9
9
  "files": [
10
10
  "bin",
11
- "plugin/skills"
11
+ "plugin/skills",
12
+ "plugin/.claude-plugin"
12
13
  ],
13
14
  "scripts": {
14
15
  "test": "python3 scripts/check.py && node scripts/test-cli.mjs"
Binary file
@@ -0,0 +1,20 @@
1
+ {
2
+ "name": "ticket-to-pr",
3
+ "version": "1.11.0",
4
+ "description": "Parallel-batch spec-driven feature workflow (spec-setup, spec-investigate, spec-create, spec-implement, spec-verify, spec-review, spec-checkpoint, spec-ship) plus a portable SEO/GEO operations skill (generative-seo) for any codebase.",
5
+ "author": {
6
+ "name": "Benjamin Thomas",
7
+ "url": "https://github.com/benjaminthomas"
8
+ },
9
+ "keywords": [
10
+ "spec-driven-development",
11
+ "feature-workflow",
12
+ "parallel-agents",
13
+ "seo",
14
+ "geo",
15
+ "productivity"
16
+ ],
17
+ "homepage": "https://github.com/benjaminthomas/spec-driven-dev",
18
+ "repository": "https://github.com/benjaminthomas/spec-driven-dev",
19
+ "icon": ".claude-plugin/icon.png"
20
+ }
@@ -67,7 +67,9 @@ Skip this step unless the changes belong to a spec (`{specs_dir}/{feature}/` exi
67
67
  references/context-template.md inside the installed `spec-setup` skill folder (not this repo): dated, with a source, added at the end, within the size
68
68
  limits, superseded lines moved to `context/archive.md`.
69
69
  - Show the context changes in the report so the user can correct them.
70
- 3. No CONTEXT.md? Skip the distilling and suggest running `spec-setup` to create one.
70
+ 3. If `risk.enabled` is on and the CLI is available, re-run `spec-driven-dev risk` so the scores
71
+ include this work. It's git-only, so it costs no tokens.
72
+ 4. No CONTEXT.md? Skip the distilling and suggest running `spec-setup` to create one.
71
73
 
72
74
  ### Step 3: Stage Everything
73
75
 
@@ -45,6 +45,18 @@ the answers to `.spec-driven.yml` (keys: `layout`, `repos`, `tracker`, `git`, `c
45
45
  - **Start the memory.** Create or extend `log.md` (`references/log-template.md`) and fill the
46
46
  README's **Resume here** block.
47
47
 
48
+ ## Risk signal (GitRisk)
49
+
50
+ Same rule as `spec-investigate`: use it only when `risk.enabled` is on (or absent) and
51
+ `{specs_dir}/context/risk.md` exists, and ignore risk completely when it's `off`. For each task that
52
+ creates or modifies a **high** or **critical** file (from the investigation's **Risk Hotspots**, or
53
+ `spec-driven-dev risk --files=…`):
54
+
55
+ - **Raise its tier by one** (light → standard → deep), and say why in Complexity's Signals.
56
+ - Fill the task's **Risk** section: the file, its score, and the past failure pattern.
57
+ - Its **Tests** must include at least one test aimed at that past failure pattern, not just the new
58
+ behaviour.
59
+
48
60
  ## When to Use
49
61
 
50
62
  - After a planning conversation where requirements and technical details have been discussed
@@ -35,6 +35,12 @@ pending
35
35
 
36
36
  **Context from dependencies:** {Prose summary of what the dependency tasks produce that this task needs. For example: "task-01-setup-database.md creates the PostgreSQL schema with users and sessions tables. This task builds the API routes that query those tables." This is what makes each file self-contained — the agent does not need to read other task files to understand its inputs.}
37
37
 
38
+ ## Risk
39
+
40
+ {Only when the task touches a high/critical file and risk.enabled is on; otherwise write "None".}
41
+
42
+ - `path/to/file.ts`: {score} {tier}. Past failures: {what the fix commits show keeps breaking}. Test: {the test in Tests aimed at it}
43
+
38
44
  ## Project Context
39
45
 
40
46
  {Only the 3–5 lines from the project context (CONTEXT.md / context/<area>.md, via the investigation's Relevant Context) that apply to THIS task: conventions, pitfalls, off-limits areas. The coder never reads the context files itself. Write "None" if nothing applies.}
@@ -25,7 +25,8 @@ You are reviewing the combined output of Batch {batch_number} of a feature imple
25
25
  - No duplicate or conflicting definitions
26
26
  - Shared state (env vars, config) is consistent
27
27
  4. **Code Quality**: Check for security issues, performance problems, and adherence to project conventions.
28
- 5. **Scope**: Verify each task stayed within its defined scope (Files to Create/Modify). Flag any unexpected file changes.
28
+ 5. **Risk**: for every task with a filled **Risk** section, check the code handles the past failure pattern it names, and that the test aimed at it exists and would fail without the fix.
29
+ 6. **Scope**: Verify each task stayed within its defined scope (Files to Create/Modify). Flag any unexpected file changes.
29
30
 
30
31
  ## Verdict
31
32
 
@@ -65,6 +65,22 @@ Cheap sources first, code second:
65
65
 
66
66
  No CONTEXT.md? Investigate as usual, and suggest running `spec-setup` afterwards to create one.
67
67
 
68
+ ## Risk signal (GitRisk)
69
+
70
+ Use it only when `.spec-driven.yml` has `risk.enabled: on`, or has no `risk:` block, **and**
71
+ `{specs_dir}/context/risk.md` exists. If `risk.enabled: off`, ignore risk entirely: don't read the
72
+ file, compute scores, or mention risk. That keeps on/off comparisons (`spec-driven-dev risk-eval`)
73
+ clean.
74
+
75
+ 1. Once you know the change surface, look up its files: `spec-driven-dev risk --files=a,b` (or
76
+ `npx -y @benjaminthomas/spec-driven-dev risk --files=…`), or find them in `risk.md`. Without the
77
+ CLI, approximate a file's history with `git log --since="2 years ago" --format=%s -- <file>`.
78
+ 2. For each **high** or **critical** file in the surface, look harder: read its last ~10 commit
79
+ subjects and the diffs of its last 1–2 fix commits, and check its dependents. Write what keeps
80
+ breaking there in the report's **Risk Hotspots** section.
81
+ 3. A score is a reason to look harder, never evidence. Hotspot findings still need their
82
+ VERIFIED / INFERRED labels.
83
+
68
84
  ## Invocation
69
85
 
70
86
  ```
@@ -22,6 +22,11 @@ Copy this structure into `specs/{feature-slug}/investigation.md`.
22
22
  - Conventions / decisions / pitfalls that apply: (one line each, with the source)
23
23
  - Context corrections: (where the code contradicts CONTEXT.md, with evidence; or "none")
24
24
 
25
+ ## Risk Hotspots
26
+ <!-- Only when risk.enabled is on. One row per high/critical file in the change surface; else "none" or omit. -->
27
+ | File | Score / tier | What has broken here before (from fix commits) | What to watch in this change |
28
+ |---|---|---|---|
29
+
25
30
  ## Business Context
26
31
  ## Problem Statement
27
32
  ## Expected Behaviour
@@ -74,7 +74,11 @@ review on a weaker model than the code was written with misses things.
74
74
  3. **The code.** Correctness (logic, edge cases, error paths, races), security (injection, authz,
75
75
  secrets, unsafe input), data safety (migrations, deletes, irreversible writes), tests (behaviour
76
76
  without a test, tests weakened, skipped or deleted), and performance on hot paths only.
77
- 4. **Run the checks.** Run `commands.lint`, `typecheck`, and `test` once if verify didn't just run
77
+ **Risk.** When `risk.enabled` is on and `{specs_dir}/context/risk.md` exists, look up the changed
78
+ files (`spec-driven-dev risk --files=…`). Give **high** and **critical** files a line-by-line read
79
+ of their error paths and callers, check that the tests target their past failure patterns, and
80
+ put the score in any finding on them. When risk is `off`, skip this and don't mention it.
81
+ 4. **Run the checks.**4. **Run the checks.** Run `commands.lint`, `typecheck`, and `test` once if verify didn't just run
78
82
  them on this exact commit. A failing check is a blocker.
79
83
 
80
84
  **Verify every finding before reporting it.** Re-read the code around it and ask "what concrete
@@ -8,7 +8,7 @@ description: >
8
8
  plus one round of questions, for a new one the intended stack and conventions, saved to a small
9
9
  specs/CONTEXT.md that later skills load cheaply. Use when the user says "set up spec-driven",
10
10
  "configure the pipeline", "spec-setup", "audit this repo", "refresh the project context",
11
- "change our branch rules", or when any other spec-* skill finds no .spec-driven.yml or no
11
+ "change our branch rules", "turn GitRisk on/off", "turn on run-through", or when any other spec-* skill finds no .spec-driven.yml or no
12
12
  CONTEXT.md. Detects everything it can first and only asks about what is ambiguous. Writes config
13
13
  and context files only — never changes code, branches, or commits.
14
14
  license: MIT
@@ -118,13 +118,33 @@ Skip if `{specs_dir}/CONTEXT.md` already exists (offer a refresh instead, see `r
118
118
  intended stack and conventions for a new one. One round of questions, no more.
119
119
  2. Write `{specs_dir}/CONTEXT.md` (and any area files) using `references/context-template.md`.
120
120
  Keep the index to 60 lines and area files to 150.
121
- 3. **Offer, don't do:** ask whether to add this one line to `CLAUDE.md` / `AGENTS.md` (whichever
121
+ 3. **Risk scores.** For an existing project, run `spec-driven-dev risk` (or
122
+ `npx -y @benjaminthomas/spec-driven-dev risk`) to write `{specs_dir}/context/risk.md`. It uses
123
+ git only, no model. Skip it for a new project with no history. Mention the hotspot count in the
124
+ report. Set `risk.enabled: off` only if the user doesn't want it.
125
+ 4. **Offer, don't do:** ask whether to add this one line to `CLAUDE.md` / `AGENTS.md` (whichever
122
126
  the project uses), so hosts reload the pointer after compaction and in every new session:
123
127
  `For spec work, read specs/CONTEXT.md first, then the "Resume here" block of any unfinished spec.`
124
128
  Add it only on a yes.
125
- 4. Re-run `npx -y @benjaminthomas/spec-driven-dev doctor` if available. It checks the size limits
129
+ 5. Re-run `npx -y @benjaminthomas/spec-driven-dev doctor` if available. It checks the size limits
126
130
  and flags context lines that point to files which no longer exist.
127
131
 
132
+ ## Quick switches (plain words, no YAML)
133
+
134
+ When the user asks to switch a feature on or off, change only that one line of `.spec-driven.yml`,
135
+ keep every other line and comment, and confirm in one sentence. Don't re-run discovery.
136
+
137
+ | They say | Set |
138
+ |---|---|
139
+ | "turn GitRisk off", "stop using risk scores", "ignore risky files" | `risk.enabled: off` |
140
+ | "turn GitRisk on", "use the risk scores" | `risk.enabled: on`, then run `spec-driven-dev risk` if `{specs_dir}/context/risk.md` doesn't exist yet |
141
+ | "run straight through after I approve", "fewer stops" | `pipeline.after_approval: run-through` |
142
+ | "stop after each stage" | `pipeline.after_approval: stop` |
143
+
144
+ Reply with what changed and what it means, for example: "GitRisk is now off. Investigations, plans
145
+ and reviews will ignore risk scores until you turn it back on." People who prefer a menu can run
146
+ `spec-driven-dev settings` in a terminal.
147
+
128
148
  ## Updating
129
149
 
130
150
  Re-running `spec-setup` with an existing config: load it, re-detect, and show only what changed or
@@ -75,6 +75,18 @@ effort:
75
75
  deep: high
76
76
  escalate: xhigh
77
77
 
78
+ # GitRisk: historical risk per file from git alone (`spec-driven-dev risk` writes {specs_dir}/context/risk.md).
79
+ # Risk = churn × 0.30 + bug_density × 0.35 + fan_out × 0.20 + security × 0.15 (each factor 0–100).
80
+ # The weights are a starting hypothesis; calibrate them against your own past bugs.
81
+ risk:
82
+ enabled: on # on | off — off makes every skill ignore risk (used by risk-eval's A/B)
83
+ since: "2 years ago" # how much history to score
84
+ weights:
85
+ churn: 0.30 # how often the file changes, vs the repo's 90th percentile
86
+ bug_density: 0.35 # share of its commits that were bug fixes
87
+ fan_out: 0.20 # how many other files import it, vs the repo's 90th percentile
88
+ security: 0.15 # share of its commits that were security fixes
89
+
78
90
  pipeline:
79
91
  # stop — every stage waits for you (default)
80
92
  # run-through — after you approve a spec: implement → verify → review → checkpoint in one go,