tewip-ai 0.29.0 → 0.30.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -552,6 +552,42 @@ With `--json` the verdict is **sealed**: editing `not-ready` to `ready` in the f
552
552
  fail, so a verdict that travels into a ticket or a release note can be checked rather than trusted. It is
553
553
  also an MCP tool, so an agent can ask whether its own work is done before saying that it is.
554
554
 
555
+ ## Run only the tests that could notice this change (Phase 25)
556
+
557
+ ```bash
558
+ tewip pick --diff main...HEAD # or: tewip pick src/pricing.js
559
+ tewip pick --list # just the files, for piping into your runner
560
+ ```
561
+
562
+ A forty-minute suite is a suite people stop running. Vendors sell this as a prediction - a model guesses
563
+ which tests matter - and a guess is the wrong shape for the job, because the cost of being wrong is a bug
564
+ reaching production and nobody can audit a guess.
565
+
566
+ Here, **skipping a test is a claim that it could not have noticed the change**, and a claim needs
567
+ evidence. A test runs when it imports a changed file, reaches one through another module, is edited by the
568
+ change itself, reads the repository at run time (a guard that scans every source file can be broken by any
569
+ file), or has failed recently — an unknown is never a skip. Everything else is left out with the reason
570
+ printed beside it.
571
+
572
+ **It refuses to narrow at all** when the change touches dependencies, build or test configuration, shared
573
+ setup or fixtures; when the suite drives a browser, because what a browser test exercises is not written
574
+ in its imports; or when a changed file cannot be traced to any test, since code is reachable in ways an
575
+ import does not show.
576
+
577
+ **Measured by breaking the code** (`eval/pick-safety.ts`): make a real mutation, ask `pick` what could
578
+ notice it, then run the **whole** suite to find out which tests really catch it. A failure is dropping a
579
+ catching test.
580
+
581
+ | Project | Judged cases | Catching test lost | Suite selected |
582
+ | --- | --- | --- | --- |
583
+ | This repository (node --test, 76 test files) | 8 | **0** | 7–9% |
584
+ | expressjs/express (Mocha, 88 test files) | 2 | **0** | refused to narrow — every test reaches the change |
585
+
586
+ Those two rows are the point. On a layered codebase it runs under a tenth of the suite and keeps every
587
+ test that would have caught the bug; on a monolith where every test imports the package front door it says
588
+ there is nothing to skip, rather than inventing a saving. It is also an MCP tool, so an agent can ask what
589
+ to run before running anything.
590
+
555
591
  ## Would your tests catch a bug? (Phase 23)
556
592
 
557
593
  ```bash
package/dist/cli/help.js CHANGED
@@ -118,6 +118,15 @@ Usage:
118
118
  is what was asked for covered. Answers ready, not ready, or cannot
119
119
  say - and cannot say is never rounded up to ready. --json writes a
120
120
  sealed verdict that can be checked later. No model, no cost.
121
+ tewip pick [--diff main...HEAD] [--list] [--out picked.md]
122
+ Which tests could notice this change? Narrows a test run using
123
+ facts rather than a prediction: a test runs when it imports a
124
+ changed file or reaches one through another module, when the change
125
+ edits the test itself, or when it failed recently - an unknown is
126
+ never a skip. It runs the whole suite when the change touches
127
+ dependencies, build or test configuration, shared setup or
128
+ fixtures, or when a changed file cannot be traced to any test.
129
+ --list prints the files for piping. No model, no cost.
121
130
  tewip prove <file> [--diff main...HEAD] [--limit 6] [--budget 600] [--strict]
122
131
  What would your tests NOT catch? Breaks your code in small,
123
132
  deliberate ways - a boundary moved by one, a condition inverted,
package/dist/cli/tewip.js CHANGED
@@ -199,6 +199,19 @@ async function main() {
199
199
  });
200
200
  return;
201
201
  }
202
+ case 'pick': {
203
+ const { runPick } = await import('../engine/pick/pick-command.js');
204
+ runPick({
205
+ root: process.cwd(),
206
+ diff: flag(args, 'diff') ?? 'main...HEAD',
207
+ files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out' && args[i - 1] !== '--recent' && args[i - 1] !== '--name'),
208
+ out: flag(args, 'out'),
209
+ list: args.includes('--list'),
210
+ recentRuns: Number(flag(args, 'recent') ?? '5'),
211
+ name: flag(args, 'name'),
212
+ });
213
+ return;
214
+ }
202
215
  case 'prove': {
203
216
  const paths = args.filter((a, i) => !a.startsWith('--') && !['--out', '--diff', '--limit', '--budget'].includes(args[i - 1] ?? ''));
204
217
  const { runProve } = await import('../engine/prove/prove-command.js');
@@ -0,0 +1,198 @@
1
+ // `tewip pick --diff main...HEAD`: the tests that could notice a change, and the command that runs them.
2
+ //
3
+ // Everything here is a fact read from the project: the diff from git, the test files on disk, the import
4
+ // graph followed outwards from each changed file, and the failures Tewip has kept. Nothing is predicted.
5
+ import { spawnSync } from 'node:child_process';
6
+ import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
7
+ import { dirname, join, relative, resolve } from 'node:path';
8
+ import { chooseFramework, detectFrameworks } from '../quality/frameworks.js';
9
+ import { changeSummary } from '../quality/grounding.js';
10
+ import { coverageFor, testFiles } from '../review/coverage.js';
11
+ import { importingTests } from '../prove/prove.js';
12
+ import { allTestsReaching } from '../prove/reaches.js';
13
+ import { datasetFiles, readJsonl, readTriageRecords } from '../../reporters/store.js';
14
+ import { pick, renderPick } from './pick.js';
15
+ const slash = (p) => p.split(String.fromCharCode(92)).join('/');
16
+ /** Tests that failed in the most recent runs, by the file they live in. */
17
+ function recentFailures(root, only, howMany) {
18
+ const dir = join(root, '.tewip', 'history');
19
+ const byFile = new Map();
20
+ if (!existsSync(dir))
21
+ return { byFile, noHistory: true };
22
+ const runs = [];
23
+ const records = [];
24
+ for (const project of only ? [only] : readdirSync(dir)) {
25
+ const where = join(dir, project);
26
+ if (!existsSync(where))
27
+ continue;
28
+ runs.push(...readJsonl(datasetFiles(where).runs));
29
+ records.push(...readTriageRecords(datasetFiles(where).triage));
30
+ }
31
+ if (runs.length === 0)
32
+ return { byFile, noHistory: true };
33
+ const recent = new Set(runs
34
+ .slice()
35
+ .sort((a, b) => (a.finishedAt > b.finishedAt ? -1 : 1))
36
+ .slice(0, howMany)
37
+ .map((r) => r.runId));
38
+ for (const record of records) {
39
+ if (!recent.has(record.runId))
40
+ continue;
41
+ const file = slash(String(record.file ?? ''));
42
+ if (!file)
43
+ continue;
44
+ byFile.set(file, [...(byFile.get(file) ?? []), record.testId]);
45
+ }
46
+ return { byFile, noHistory: false };
47
+ }
48
+ /** A test file's path as the history records it, which may be a bare name rather than a full path. */
49
+ const sameTest = (testFile, recorded) => testFile === recorded || testFile.endsWith(`/${recorded}`) || recorded.endsWith(`/${testFile}`);
50
+ /**
51
+ * Does this test walk the repository at run time?
52
+ *
53
+ * Such a test - a guard that scans every source file, a check on licence headers, a snapshot of the
54
+ * public API - can be broken by any file in the project, so it must never be skipped. This is not a
55
+ * guess at coverage: it is a statement about what the test reads. Found the hard way, by a safety
56
+ * measurement that caught this project's own control-character guard failing over a change to a module
57
+ * it does not import.
58
+ */
59
+ function readsTheRepo(root, testFile) {
60
+ try {
61
+ const text = readFileSync(resolve(root, testFile), 'utf8');
62
+ const walks = /\breaddirSync\b|\bglob(?:Sync)?\s*\(|\bwalk\s*\(|os\.walk|Files\.walk|filepath\.Walk/.test(text);
63
+ const reads = /\breadFileSync\b|\bopen\s*\(|readFile\b/.test(text);
64
+ return walks && reads;
65
+ }
66
+ catch {
67
+ return false;
68
+ }
69
+ }
70
+ /**
71
+ * One command rather than a chain of them. Runners take several files at once, and a line of forty
72
+ * `&&`-joined invocations pays the start-up cost forty times - which is most of the saving the selection
73
+ * was for. The shared prefix is taken from the framework's own command for two files, so nothing is
74
+ * assumed about how it is spelled.
75
+ */
76
+ function oneCommand(root, picked, framework) {
77
+ if (!framework)
78
+ return 'your test command';
79
+ if (picked.mustRun.length === 0)
80
+ return framework.suite;
81
+ const paths = picked.mustRun.map((m) => resolve(root, m.testFile));
82
+ const first = framework.command(paths[0]);
83
+ if (paths.length === 1)
84
+ return withCwd(root, framework.cwd, first);
85
+ // Everything before the first quoted argument is the runner. A longest-common-prefix of two commands
86
+ // looked cleverer and cut inside the first file name, producing `node --test "test/ "test/a.ts"`.
87
+ const quote = first.indexOf('"');
88
+ const prefix = quote > 0 ? first.slice(0, quote).trimEnd() : '';
89
+ if (!prefix)
90
+ return withCwd(root, framework.cwd, picked.mustRun.map((m) => framework.command(resolve(root, m.testFile))).join(' && '));
91
+ const quoted = paths.map((p) => JSON.stringify(relative(framework.cwd, p).split(String.fromCharCode(92)).join('/'))).join(' ');
92
+ return withCwd(root, framework.cwd, `${prefix} ${quoted}`);
93
+ }
94
+ /** Says where to run it, when that is not the project root. */
95
+ const withCwd = (root, cwd, command) => (resolve(cwd) === resolve(root) ? command : `cd ${JSON.stringify(relative(root, cwd).split(String.fromCharCode(92)).join('/'))} && ${command}`);
96
+ /** Test files that mention this file by path or by name, which is weaker than an import but not nothing. */
97
+ function mentionsOf(root, file, tests) {
98
+ // The name **with its extension**, or the path. Matching the bare stem was useless: for `cli/tewip.ts`
99
+ // the stem is "tewip", which appears in nearly every file of this project, so 28 test files were
100
+ // selected on the grounds that they had mentioned the product's own name.
101
+ const named = file.split('/').pop() ?? '';
102
+ if (named.length < 5)
103
+ return [];
104
+ const out = [];
105
+ for (const test of tests) {
106
+ try {
107
+ const text = readFileSync(resolve(root, test), 'utf8');
108
+ if (text.includes(file) || text.includes(named))
109
+ out.push(test);
110
+ }
111
+ catch {
112
+ // Unreadable: it cannot be shown to mention anything.
113
+ }
114
+ }
115
+ return out;
116
+ }
117
+ /** Does anything at all in the project refer to this file? Used only to tell "unknown" from "unreachable". */
118
+ function mentionedAnywhere(root, file) {
119
+ const named = file.split('/').pop() ?? '';
120
+ const where = spawnSync('git', ['grep', '-l', '--', named], { cwd: root, encoding: 'utf8', timeout: 60_000 });
121
+ const hits = (where.stdout ?? '')
122
+ .split(String.fromCharCode(10))
123
+ .map((l) => l.trim())
124
+ .filter((l) => l && l.split(String.fromCharCode(92)).join('/') !== file);
125
+ return hits.length > 0;
126
+ }
127
+ export function choose(cmd) {
128
+ const change = cmd.files && cmd.files.length > 0 ? { files: cmd.files } : changeSummary(cmd.root, cmd.diff);
129
+ if (!change)
130
+ return null;
131
+ const changed = change.files.map(slash);
132
+ const known = testFiles(cmd.root).map(slash);
133
+ const relative = known.map((t) => slash(t).replace(`${slash(cmd.root)}/`, ''));
134
+ const { byFile, noHistory } = recentFailures(cmd.root, cmd.name, cmd.recentRuns ?? 5);
135
+ const candidates = relative.map((testFile) => {
136
+ const failed = [];
137
+ for (const [file, ids] of byFile)
138
+ if (sameTest(testFile, file))
139
+ failed.push(...ids);
140
+ return { testFile, recentlyFailed: failed, readsTheRepo: readsTheRepo(cmd.root, testFile) };
141
+ });
142
+ // The import graph, followed outwards from each changed source file.
143
+ const reaching = new Map();
144
+ const sources = changed.filter((f) => existsSync(resolve(cmd.root, f)));
145
+ for (const file of sources) {
146
+ // Both kinds, together: a test may import the file itself and others reach it through the package.
147
+ // Taking only the direct importers when there are any is how a selection quietly drops the tests
148
+ // that exercise the code through the application's own front door.
149
+ const direct = importingTests(cmd.root, coverageFor(cmd.root, [file], testFiles(cmd.root)), file);
150
+ const reached = allTestsReaching(cmd.root, file, relative);
151
+ const byFile = new Map();
152
+ for (const r of reached)
153
+ byFile.set(slash(r.testFile), { testFile: slash(r.testFile), how: `reaches, through ${r.steps} import(s),` });
154
+ for (const testFile of direct)
155
+ byFile.set(slash(testFile), { testFile: slash(testFile), how: 'imports' });
156
+ reaching.set(file, [...byFile.values()]);
157
+ }
158
+ // A file nothing imports may still be named somewhere - a test that reads it as text, a path in a
159
+ // script, a dynamic import. A mention is not proof that a test exercises it, so a test that mentions
160
+ // it runs; and when nothing in the project mentions it at all, nothing can notice it.
161
+ const unreferenced = [];
162
+ for (const [file, reaches] of reaching) {
163
+ if (reaches.length > 0)
164
+ continue;
165
+ const named = mentionsOf(cmd.root, file, relative);
166
+ if (named.length > 0)
167
+ reaching.set(file, named.map((testFile) => ({ testFile, how: 'names' })));
168
+ else if (!mentionedAnywhere(cmd.root, file))
169
+ unreferenced.push(file);
170
+ }
171
+ const framework = chooseFramework(detectFrameworks(cmd.root), undefined, false) ?? detectFrameworks(cmd.root)[0] ?? null;
172
+ const picked = pick({ changed, tests: candidates, reaching, browserTests: Boolean(framework?.browser), noHistory, unreferenced });
173
+ const command = oneCommand(cmd.root, picked, framework);
174
+ return { picked, total: candidates.length, command, noHistory };
175
+ }
176
+ export function runPick(cmd) {
177
+ const chosen = choose(cmd);
178
+ if (!chosen)
179
+ throw new Error(`could not read the change "${cmd.diff}". Is this a git checkout, and does that range exist? (try --diff main...HEAD)`);
180
+ if (cmd.list) {
181
+ // For piping into a runner: the files, and nothing else. A run-everything decision prints nothing,
182
+ // so a script that pipes this cannot mistake "cannot narrow" for "no tests to run".
183
+ if (chosen.picked.runEverything) {
184
+ console.error(`tewip pick: ${chosen.picked.runEverything}. Run the whole suite.`);
185
+ process.exitCode = 2;
186
+ return;
187
+ }
188
+ for (const must of chosen.picked.mustRun)
189
+ console.log(must.testFile);
190
+ return;
191
+ }
192
+ const report = renderPick(chosen.picked, chosen.total, chosen.command, chosen.noHistory);
193
+ console.log(report);
194
+ if (cmd.out) {
195
+ mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
196
+ writeFileSync(resolve(cmd.out), report + String.fromCharCode(10), 'utf8');
197
+ }
198
+ }
@@ -0,0 +1,117 @@
1
+ /** A changed file that can affect anything, so no test can be ruled out because of it. */
2
+ const GLOBAL = [
3
+ { match: /(^|\/)(package|package-lock|yarn|pnpm-lock|bun\.lock|requirements|pyproject|poetry\.lock|go\.mod|go\.sum|pom\.xml|build\.gradle|Gemfile|composer\.json|Cargo\.toml)/i, why: 'it changes the dependencies, which every test runs against' },
4
+ { match: /(^|\/)(\.github|\.gitlab-ci|Jenkinsfile|azure-pipelines|\.circleci|Dockerfile|docker-compose|Makefile)/i, why: 'it changes how the suite is built or run' },
5
+ { match: /(^|\/)(jest|vitest|playwright|cypress|karma|babel|tsconfig|webpack|vite|rollup|eslint|tox|pytest|conftest)\b[\w.]*\.(json|js|ts|cjs|mjs|ini|cfg|toml|yaml|yml)$/i, why: 'it is test or build configuration, which applies to the whole suite' },
6
+ { match: /(^|\/)(conftest\.py|setup\.(js|ts|py)|global-setup\.[jt]s|jest\.setup\.[jt]s|test-setup\.[jt]s)$/i, why: 'it is shared setup, which runs before every test' },
7
+ { match: /(^|\/)(fixtures?|helpers?|support|mocks?|__mocks__|testdata|factories)\//i, why: 'it is shared test support, which any test may use' },
8
+ ];
9
+ const SOURCE = /\.(js|jsx|mjs|cjs|ts|tsx|mts|cts|py|java|kt|go|cs|rb|php|vue|svelte|scala|swift)$/i;
10
+ const IS_TEST = /(^|\/)(tests?|spec|specs|__tests__|e2e)\/|[._-](test|tests|spec|specs)\.[a-z]+$|(^|\/)test_[^/]+\.py$|_test\.(go|py|rb)$|Tests?\.(java|kt|cs)$/i;
11
+ /** Documentation and assets: a test cannot notice these, so they never widen the run. */
12
+ const INERT = /\.(md|mdx|txt|rst|png|jpe?g|gif|svg|ico|webp|woff2?|ttf|eot|mp4|pdf|lock|csv)$|(^|\/)(LICENSE|CODEOWNERS|\.gitignore|\.editorconfig|\.prettierrc)/i;
13
+ export function pick(input) {
14
+ // Global first, then inert. The other order called `requirements.txt` documentation because of its
15
+ // extension and then reported that a dependency change could not affect any test.
16
+ const changed = input.changed.filter((f) => GLOBAL.some((g) => g.match.test(f)) || !INERT.test(f));
17
+ const picked = { mustRun: [], skipped: [], runEverything: '', opaque: [] };
18
+ if (changed.length === 0) {
19
+ return { ...picked, runEverything: 'nothing in this change can affect a test result - only documentation, assets or lockless files were touched. Running nothing is also an option.' };
20
+ }
21
+ // Anything global stops selection outright, and names the file that did it.
22
+ for (const file of changed) {
23
+ const global = GLOBAL.find((g) => g.match.test(file));
24
+ if (global)
25
+ return { ...picked, runEverything: `${file} cannot be narrowed around: ${global.why}` };
26
+ }
27
+ const changedSource = changed.filter((f) => SOURCE.test(f) && !IS_TEST.test(f));
28
+ const changedTests = changed.filter((f) => IS_TEST.test(f));
29
+ // A changed file nothing can be traced from. For a browser suite this is the normal case rather than a
30
+ // gap - the test reaches the code through the running application - so the honest answer is to run
31
+ // everything and say why, not to claim the tests are unrelated.
32
+ // A file no test imports may still be reachable in a way imports do not show. But a file that nothing
33
+ // in the project refers to **at all** - not imported, not named in any source or test - cannot be
34
+ // noticed by any test, and treating that as "cannot tell" made the command useless: in this very
35
+ // repository a change to the command-line dispatcher, which tests only read as text, forced the whole
36
+ // suite. So the two cases are separated, and only a genuine unknown stops selection.
37
+ picked.opaque = changedSource.filter((f) => (input.reaching.get(f) ?? []).length === 0 && !input.unreferenced.includes(f));
38
+ if (picked.opaque.length > 0) {
39
+ return {
40
+ ...picked,
41
+ runEverything: input.browserTests
42
+ ? `these tests drive a browser, so what a test exercises is not written in its imports: ${picked.opaque.slice(0, 3).join(', ')} cannot be traced to any test, and skipping on that basis would be a guess`
43
+ : `no test can be traced to ${picked.opaque.slice(0, 3).join(', ')}${picked.opaque.length > 3 ? ` and ${picked.opaque.length - 3} more` : ''}. Tewip will not take that as proof the suite is unaffected - code can be reached in ways an import does not show (dependency injection, a plugin registry, a dynamic import)`,
44
+ };
45
+ }
46
+ const why = new Map();
47
+ const add = (testFile, reason) => {
48
+ if (!why.has(testFile))
49
+ why.set(testFile, reason);
50
+ };
51
+ // A test the change itself edited always runs: whatever it now checks, nobody has seen it run yet.
52
+ for (const file of changedTests)
53
+ add(file, 'this change edits it');
54
+ for (const [file, reaches] of input.reaching) {
55
+ if (!changedSource.includes(file))
56
+ continue;
57
+ for (const reach of reaches)
58
+ add(reach.testFile, `it ${reach.how} ${file}`);
59
+ }
60
+ // A test that walks the repository can be broken by any file in it, so it always runs.
61
+ for (const test of input.tests) {
62
+ if (test.readsTheRepo)
63
+ add(test.testFile, 'it reads the repository itself, so any changed file can break it');
64
+ }
65
+ // An unknown is not a skip. A test that has been failing or flaking lately gets another run, because
66
+ // "it is unrelated to this change" and "it is broken for another reason" are different statements and
67
+ // only the first is a reason to skip.
68
+ for (const test of input.tests) {
69
+ if (test.recentlyFailed.length > 0)
70
+ add(test.testFile, `it failed recently (${test.recentlyFailed.length} time(s)), so its state is unknown rather than unrelated`);
71
+ }
72
+ picked.mustRun = [...why.entries()].map(([testFile, reason]) => ({ testFile, why: reason })).sort((a, b) => a.testFile.localeCompare(b.testFile));
73
+ picked.skipped = input.tests
74
+ .filter((t) => !why.has(t.testFile))
75
+ .map((t) => ({ testFile: t.testFile, why: 'nothing it imports, directly or through another module, reaches anything this change touched' }))
76
+ .sort((a, b) => a.testFile.localeCompare(b.testFile));
77
+ if (picked.mustRun.length === 0) {
78
+ return {
79
+ ...picked,
80
+ runEverything: `nothing in this project refers to ${changed.slice(0, 3).join(', ')}, so no test can notice the change. Running nothing is defensible; running the suite costs nothing but time.`,
81
+ };
82
+ }
83
+ // Selecting everything is not a failure, but it should be said plainly rather than dressed up as a saving.
84
+ if (picked.skipped.length === 0 && picked.mustRun.length > 0) {
85
+ picked.runEverything = 'every test in the project reaches something this change touched, so there is nothing to skip';
86
+ }
87
+ if (input.noHistory && picked.skipped.length > 0) {
88
+ picked.runEverything = picked.runEverything || '';
89
+ }
90
+ return picked;
91
+ }
92
+ /** What the selection is worth, and what it rests on. */
93
+ export function renderPick(picked, total, command, noHistory) {
94
+ const lines = ['# Which tests could notice this change?', ''];
95
+ if (picked.runEverything) {
96
+ lines.push(`**Run the whole suite.** ${picked.runEverything}.`, '', 'Tewip narrows a test run only when it can show that the tests it leaves out could not have noticed', 'the change. It cannot show that here, so it is not asking you to take the risk.', '');
97
+ if (picked.opaque.length > 0) {
98
+ lines.push('Changed files nothing could be traced from:', '', ...picked.opaque.slice(0, 10).map((f) => `- ${f}`), '');
99
+ }
100
+ return lines.concat(tail(noHistory)).join(String.fromCharCode(10));
101
+ }
102
+ const kept = picked.mustRun.length;
103
+ const share = total > 0 ? Math.round((kept / total) * 100) : 100;
104
+ lines.push(`**${kept} of ${total} test file(s)** could notice it - ${share}% of the suite. ${picked.skipped.length} can be left out.`, '', '```', command, '```', '', `## Why these ${kept} run`, '', ...picked.mustRun.slice(0, 30).map((m) => `- \`${m.testFile}\` — ${m.why}`), ...(picked.mustRun.length > 30 ? [`- …and ${picked.mustRun.length - 30} more`] : []), '', `## What you are giving up (${picked.skipped.length})`, '', 'Each of these is left out because of a fact about the source, not a prediction:', '', ...picked.skipped.slice(0, 15).map((s) => `- \`${s.testFile}\``), ...(picked.skipped.length > 15 ? [`- …and ${picked.skipped.length - 15} more`] : []), '');
105
+ return lines.concat(tail(noHistory)).join(String.fromCharCode(10));
106
+ }
107
+ const tail = (noHistory) => [
108
+ '---',
109
+ '',
110
+ 'How this decides: a test runs when it imports a changed file, or reaches one through another module,',
111
+ 'or when the change edits the test itself, or when it has failed recently - an unknown is never a skip.',
112
+ 'It runs everything when the change touches dependencies, build or test configuration, shared setup or',
113
+ 'shared fixtures, or when a changed file cannot be traced to any test at all. No model is called.',
114
+ ...(noHistory
115
+ ? ['', 'This project keeps no run history, so "it has not failed lately" could not be checked for any test.', 'Run `tewip explain` on your CI reports and the next selection will know more.']
116
+ : []),
117
+ ];
@@ -24,10 +24,14 @@ const NOT_BEHAVIOUR = /^\s*(\/\/|#|\*|\/\*|@|import\b|from\b|export\s+(type|inte
24
24
  * sign the wrong way round, a flag returned the wrong way.
25
25
  */
26
26
  const OPERATORS = [
27
- { kind: 'a boundary moved by one', find: /(?<![<>=!])<=(?!=)/, swap: () => '<' },
28
- { kind: 'a boundary moved by one', find: /(?<![<>=!])>=(?!=)/, swap: () => '>' },
29
- { kind: 'a boundary moved by one', find: /(?<![<>=!+-])<(?![<=])/, swap: () => '<=' },
30
- { kind: 'a boundary moved by one', find: /(?<![<>=!+-])>(?![>=])/, swap: () => '>=' },
27
+ // Whitespace on both sides, which is what separates a comparison from a generic type. Without it,
28
+ // `Array<{ id: string }>` was read as a comparison and mutated into `Array<={ … }>`: the file stopped
29
+ // compiling, the whole suite failed to load, and the measurement counted a mutant nothing caught.
30
+ // A codebase that writes `a<b` loses a few mutants; one that breaks its build loses the answer.
31
+ { kind: 'a boundary moved by one', find: /(?<=\s)<=(?=\s)/, swap: () => '<' },
32
+ { kind: 'a boundary moved by one', find: /(?<=\s)>=(?=\s)/, swap: () => '>' },
33
+ { kind: 'a boundary moved by one', find: /(?<=\s)<(?=\s)/, swap: () => '<=' },
34
+ { kind: 'a boundary moved by one', find: /(?<=\s)>(?=\s)/, swap: () => '>=' },
31
35
  { kind: 'a comparison inverted', find: /===/, swap: () => '!==' },
32
36
  { kind: 'a comparison inverted', find: /(?<![!=<>])==(?!=)/, swap: () => '!=' },
33
37
  { kind: 'a comparison inverted', find: /!==/, swap: () => '===' },
@@ -80,10 +80,11 @@ export function proveFile(root, file, covering, framework, opts = {}) {
80
80
  return finish(proof, started);
81
81
  }
82
82
  const original = readFileSync(full, 'utf8');
83
+ const stopGuarding = guardAgainstBeingKilled(full, original);
83
84
  const mutants = mutantsFor(original, limit);
84
85
  if (mutants.length === 0) {
85
86
  proof.blocked = 'nothing in this file could be changed in a way a test should be expected to catch - no comparison, no condition, no returned value outside of strings and comments';
86
- return finish(proof, started);
87
+ return finish(proof, started, stopGuarding);
87
88
  }
88
89
  // A test has to pass before it can be asked to notice anything, or a red suite turns into "your
89
90
  // tests catch nothing", which is the worst kind of lie this command could tell.
@@ -103,7 +104,7 @@ export function proveFile(root, file, covering, framework, opts = {}) {
103
104
  }
104
105
  if (usable.length === 0) {
105
106
  proof.blocked = `none of the ${covering.length} test file(s) covering this pass as they are, so none can be asked whether they would notice a change. Fix the run first.`;
106
- return finish(proof, started);
107
+ return finish(proof, started, stopGuarding);
107
108
  }
108
109
  proof.covering = usable;
109
110
  // The control: break the file outright and check that a covering test notices. Without this, a file
@@ -111,7 +112,7 @@ export function proveFile(root, file, covering, framework, opts = {}) {
111
112
  say('checking the tests really load this file');
112
113
  if (!testsLoadIt(root, full, original, usable[0], framework, runMs)) {
113
114
  proof.blocked = `${usable[0]} passes even when this file is made unusable, so the tests are not loading the file that was edited - they are reading an installed copy or a build output. Nothing here can be proved until they run against this source (for a Python src layout, install it in editable mode: pip install -e .).`;
114
- return finish(proof, started);
115
+ return finish(proof, started, stopGuarding);
115
116
  }
116
117
  // Only the tests that pass may judge a mutant. A test that is already failing fails for every
117
118
  // change too, which would have it "catch" everything - the exact false confidence this command
@@ -156,7 +157,7 @@ export function proveFile(root, file, covering, framework, opts = {}) {
156
157
  }
157
158
  proof.tried.push({ mutant, fate, noticedBy, seconds: Math.round((Date.now() - at) / 100) / 10 });
158
159
  }
159
- return finish(proof, started);
160
+ return finish(proof, started, stopGuarding);
160
161
  }
161
162
  /**
162
163
  * Puts the file back and checks that it really went back. Anything less would mean this command can
@@ -168,7 +169,43 @@ function restore(full, original) {
168
169
  throw new Error(`${full} could not be put back as it was. Check it before doing anything else: git checkout -- "${full}"`);
169
170
  }
170
171
  }
171
- const finish = (proof, started) => ({ ...proof, seconds: Math.round((Date.now() - started) / 1000) });
172
+ /**
173
+ * Put the file back even when the process is killed.
174
+ *
175
+ * `finally` does not run for SIGINT or SIGTERM, so a developer pressing Ctrl-C during a run - which they
176
+ * will, because this is the slow command - was left with a mutated source file and no idea why their
177
+ * code had stopped working. Found by killing one of these runs during development and finding a flipped
178
+ * comparison still sitting in the repository.
179
+ */
180
+ function guardAgainstBeingKilled(full, original) {
181
+ const putBack = (signal) => () => {
182
+ try {
183
+ restore(full, original);
184
+ process.stderr.write(`${String.fromCharCode(10)}${signal}: ${full} was put back as it was.${String.fromCharCode(10)}`);
185
+ }
186
+ catch {
187
+ process.stderr.write(`${String.fromCharCode(10)}${signal}: could not put ${full} back. Run: git checkout -- "${full}"${String.fromCharCode(10)}`);
188
+ }
189
+ process.exit(130);
190
+ };
191
+ const handlers = ['SIGINT', 'SIGTERM', 'SIGHUP', 'SIGBREAK'].map((signal) => [signal, putBack(signal)]);
192
+ for (const [signal, handler] of handlers) {
193
+ try {
194
+ process.on(signal, handler);
195
+ }
196
+ catch {
197
+ // Not every signal exists on every platform.
198
+ }
199
+ }
200
+ return () => {
201
+ for (const [signal, handler] of handlers)
202
+ process.off(signal, handler);
203
+ };
204
+ }
205
+ const finish = (proof, started, stopGuarding) => {
206
+ stopGuarding?.();
207
+ return { ...proof, seconds: Math.round((Date.now() - started) / 1000) };
208
+ };
172
209
  const firstLine = (output) => output.split(String.fromCharCode(10)).map((l) => l.trim()).find(Boolean)?.slice(0, 160) ?? '';
173
210
  /** Which of the tests that cover a file really import it, and so could notice a change. */
174
211
  export function importingTests(root, coverage, file) {
@@ -9,7 +9,7 @@
9
9
  // The chain is followed outwards from the file, not inwards from the tests, because the answer needed
10
10
  // is "who would notice if this changed" and that is what a reverse import graph gives.
11
11
  import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs';
12
- import { join, relative, resolve } from 'node:path';
12
+ import { dirname, join, relative, resolve } from 'node:path';
13
13
  import { importNames, importsIt } from '../review/coverage.js';
14
14
  const SKIP_DIR = new Set(['node_modules', '.git', 'dist', 'build', 'out', 'coverage', 'target', 'vendor', '__pycache__', '.venv', 'venv', '.tewip', '.next']);
15
15
  const SOURCE = /\.(?:[cm]?[jt]sx?|py|go|java|kt|cs|rb)$/i;
@@ -46,6 +46,116 @@ function sourceFiles(root, limit = 6000) {
46
46
  walk(root);
47
47
  return out;
48
48
  }
49
+ /**
50
+ * The package's front door: the files a test means when it writes `require('..')`.
51
+ *
52
+ * This matters for selection and nothing else. express's tests all do `require('..')`, which resolves
53
+ * through `index.js` to `lib/express.js` and from there to every part of the library - so a change to
54
+ * `lib/response.js` really is reachable from almost every test in the project. Without this, selection
55
+ * followed the chain as far as `index.js`, found that no test imports *that* by name, and cheerfully
56
+ * declared 86 of express's 88 test files unaffected by a change to the response code.
57
+ */
58
+ function frontDoor(root) {
59
+ const files = new Set();
60
+ try {
61
+ const manifest = JSON.parse(readFileSync(resolve(root, 'package.json'), 'utf8'));
62
+ if (manifest.main)
63
+ files.add(resolve(root, manifest.main));
64
+ }
65
+ catch {
66
+ // No manifest: the conventional entry points below still apply.
67
+ }
68
+ for (const entry of ['index.js', 'index.ts', 'index.mjs', 'index.cjs', 'index.tsx', '__init__.py', 'src/index.js', 'src/index.ts']) {
69
+ if (existsSync(resolve(root, entry)))
70
+ files.add(resolve(root, entry));
71
+ }
72
+ return files;
73
+ }
74
+ /** Every path this file imports, as written. */
75
+ const importedPaths = (text) => [...text.matchAll(/(?:from|require\s*\(|import)\s*['"]([^'"]+)['"]/g)].map((m) => m[1]);
76
+ /**
77
+ * Does this file import the package's front door?
78
+ *
79
+ * Decided by **resolving** each import against the file's own directory, not by recognising spellings.
80
+ * Guessing at spellings had already missed one: express's `test/res.status.js` imports `require('../.')`,
81
+ * which is the package root written a slightly unusual way, and the selection declared that file
82
+ * unaffected by a change to the very module `res.status` lives in. A false skip is the one failure this
83
+ * command must not have, so the question is answered by path resolution, which cannot be spelled around.
84
+ */
85
+ function importsFrontDoor(root, file, text, doors) {
86
+ const here = dirname(resolve(root, file));
87
+ for (const spec of importedPaths(text)) {
88
+ if (!spec.startsWith('.'))
89
+ continue;
90
+ const target = resolve(here, spec);
91
+ if (target === resolve(root))
92
+ return true; // the package root itself
93
+ for (const ending of ['', '.js', '.ts', '.mjs', '.cjs', '.tsx', '/index.js', '/index.ts', '/index.mjs', '/index.cjs', '/index.tsx', '/__init__.py']) {
94
+ if (doors.has(resolve(`${target}${ending}`)))
95
+ return true;
96
+ }
97
+ }
98
+ return false;
99
+ }
100
+ export function allTestsReaching(root, file, tests, depth = 6) {
101
+ const all = sourceFiles(root);
102
+ const text = new Map();
103
+ const read = (path) => {
104
+ if (!text.has(path)) {
105
+ try {
106
+ text.set(path, readFileSync(path, 'utf8'));
107
+ }
108
+ catch {
109
+ text.set(path, '');
110
+ }
111
+ }
112
+ return text.get(path);
113
+ };
114
+ const rel = (path) => relative(root, path).split(String.fromCharCode(92)).join('/');
115
+ const testSet = new Set(tests.map((t) => rel(resolve(root, t))));
116
+ const found = new Map();
117
+ const seen = new Set([rel(resolve(root, file))]);
118
+ const doors = frontDoor(root);
119
+ let layer = [file];
120
+ for (let step = 1; step <= depth && layer.length > 0; step += 1) {
121
+ // Reached the package's own entry point. Everything that imports the package now reaches the change -
122
+ // and that includes modules, not only tests: express's acceptance tests import an example app, which
123
+ // imports the package, so stopping at tests here left nineteen of them looking unaffected.
124
+ if (layer.some((f) => doors.has(resolve(root, f)))) {
125
+ const throughDoor = [];
126
+ for (const path of all) {
127
+ const where = rel(path);
128
+ if (seen.has(where) || !importsFrontDoor(root, where, read(path), doors))
129
+ continue;
130
+ seen.add(where);
131
+ if (testSet.has(where))
132
+ found.set(where, { testFile: where, steps: step, via: 'the package itself' });
133
+ else
134
+ throughDoor.push(where);
135
+ }
136
+ layer = [...layer, ...throughDoor];
137
+ }
138
+ const names = layer.flatMap((f) => (existsSync(resolve(root, f)) ? importNames(root, f) : []));
139
+ if (names.length === 0)
140
+ break;
141
+ const next = [];
142
+ for (const path of all) {
143
+ const where = rel(path);
144
+ if (seen.has(where))
145
+ continue;
146
+ if (!importsIt(read(path), names))
147
+ continue;
148
+ seen.add(where);
149
+ if (testSet.has(where))
150
+ found.set(where, { testFile: where, steps: step, via: layer[0] });
151
+ // A module that imports the change is itself a way in, whether or not it is a test, so the walk
152
+ // continues through it. This is what reaches a test that only imports the package front door.
153
+ next.push(where);
154
+ }
155
+ layer = next;
156
+ }
157
+ return [...found.values()];
158
+ }
49
159
  /**
50
160
  * The tests that reach `file`, directly or through up to `depth` modules.
51
161
  *
@@ -176,4 +176,25 @@ server.registerTool('is_the_work_done', {
176
176
  return `Nothing changed in ${range}, or that range could not be read here.`;
177
177
  return renderDecision(range, decide(gathered.evidence));
178
178
  }));
179
+ server.registerTool('which_tests_to_run', {
180
+ title: 'Which tests could notice this change?',
181
+ description: "Narrows a test run to the tests that could notice a change, from facts in the source rather than a prediction: a test runs when it imports a changed file, reaches one through another module, is itself edited by the change, reads the repository at run time, or has failed recently. It refuses to narrow at all - and says why - when the change touches dependencies, build or test configuration, shared setup or fixtures, when the suite drives a browser, or when a changed file cannot be traced to any test. Use it to run the right tests quickly; what it leaves out, it leaves out for a stated reason.",
182
+ inputSchema: {
183
+ project,
184
+ diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
185
+ files: z.array(z.string()).optional().describe('The changed files, when they do not come from a git range'),
186
+ },
187
+ annotations: readOnly,
188
+ }, ({ project: root, diff, files }) => answer(async () => {
189
+ if (!isAbsolute(root) || !existsSync(root))
190
+ throw new Error(`project must be an existing absolute path, got "${root}"`);
191
+ // choose/renderPick are pure; runPick prints, which would corrupt the protocol.
192
+ const { choose } = await import('../engine/pick/pick-command.js');
193
+ const { renderPick } = await import('../engine/pick/pick.js');
194
+ const range = diff ?? 'main...HEAD';
195
+ const chosen = choose({ root, diff: range, files });
196
+ if (!chosen)
197
+ return `Nothing changed in ${range}, or that range could not be read here.`;
198
+ return renderPick(chosen.picked, chosen.total, chosen.command, chosen.noHistory);
199
+ }));
179
200
  await server.connect(new StdioServerTransport());
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "tewip-ai",
3
- "version": "0.29.0",
3
+ "version": "0.30.0",
4
4
  "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
5
5
  "keywords": [
6
6
  "test-failures",