tewip-ai 0.29.0 → 0.31.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -552,6 +552,104 @@ With `--json` the verdict is **sealed**: editing `not-ready` to `ready` in the f
552
552
  fail, so a verdict that travels into a ticket or a release note can be checked rather than trusted. It is
553
553
  also an MCP tool, so an agent can ask whether its own work is done before saying that it is.
554
554
 
555
+ ## What will this migration do to a live database? (Phase 26)
556
+
557
+ ```bash
558
+ tewip schema --diff main...HEAD # or: tewip schema migrations/0002_drop_nickname.sql
559
+ tewip schema --strict # fails the build on a change that breaks the running version
560
+ ```
561
+
562
+ The most expensive change in a pull request is usually not a test — it is a migration. A column dropped
563
+ while the previous deployment is still selecting it. An index built without `CONCURRENTLY`, blocking every
564
+ write to the busiest table. A `NOT NULL` added to a populated table. These are known, documented, and a
565
+ tool can name every one of them from the text of the migration.
566
+
567
+ ```
568
+ 1 of 1 migration file(s) read. 1 change(s) would break the running version or destroy data
569
+
570
+ ## Breaks the version that is still running (1)
571
+ - drops the column nickname from orders — migrations/0002_drop_nickname.sql:1
572
+ - Instead: deploy code that no longer selects or writes the column, then drop it in a later
573
+ migration. During a rolling deploy the old version is still running and will fail on a column
574
+ that is gone
575
+
576
+ ## Code that still uses what this takes away (1)
577
+ - orders.nickname is still named in 1 file(s): app/orders.py
578
+ ```
579
+
580
+ That last section is the part no SQL linter does: it searches your own code for what the migration takes
581
+ away. "This drops `users.nickname`, and four files still read it" is a different sentence from "dropping a
582
+ column is risky", and it is the one that stops the outage.
583
+
584
+ It reads **SQL, Django, Rails and Alembic** migrations, judges **only the forward path** (a rollback step
585
+ contains the mirror image of the change, so reading it says a migration that creates a table destroys
586
+ one), and stays quiet when the team already did the safe thing — `atomic = False`, `algorithm:
587
+ :concurrently`, `NOT VALID`, `postgresql_concurrently=True`. When it cannot read a migration it says so
588
+ rather than passing it, and "read and nothing found" never looks like "not read".
589
+
590
+ **Tried on real projects.** 200 Zulip migrations: 199 read, 19 changes that break the running version or
591
+ destroy data, each sampled finding correct when read against the file — including a `DROP COLUMN` buried
592
+ in raw SQL inside a Python data migration. 60 Mastodon migrations: it caught a `change_column` that
593
+ rewrites a table, in a migration whose *name* says it only adds one.
594
+
595
+ What it does not know, and says so: your table sizes and how busy the database is, so it reports what a
596
+ statement **does** — rewrites the table, scans every row, blocks writes — and leaves the duration to the
597
+ person who knows the data. In CI the findings go into the pull request comment, and anything that breaks
598
+ the running version enters the evidence bundle as a refusal: a passing test suite does not verify a
599
+ migration.
600
+
601
+ ## Run only the tests that could notice this change (Phase 25)
602
+
603
+ ```bash
604
+ tewip pick --diff main...HEAD # or: tewip pick src/pricing.js
605
+ tewip pick --list # just the files, for piping into your runner
606
+ ```
607
+
608
+ A forty-minute suite is a suite people stop running. Vendors sell this as a prediction - a model guesses
609
+ which tests matter - and a guess is the wrong shape for the job, because the cost of being wrong is a bug
610
+ reaching production and nobody can audit a guess.
611
+
612
+ Here, **skipping a test is a claim that it could not have noticed the change**, and a claim needs
613
+ evidence. A test runs when it imports a changed file, reaches one through another module, is edited by the
614
+ change itself, reads the repository at run time (a guard that scans every source file can be broken by any
615
+ file), or has failed recently — an unknown is never a skip. Everything else is left out with the reason
616
+ printed beside it.
617
+
618
+ **It refuses to narrow at all** when the change touches dependencies, build or test configuration, shared
619
+ setup or fixtures; when the suite drives a browser, because what a browser test exercises is not written
620
+ in its imports; or when a changed file cannot be traced to any test, since code is reachable in ways an
621
+ import does not show.
622
+
623
+ **Measured by breaking the code** (`eval/pick-safety.ts`): make a real mutation, ask `pick` what could
624
+ notice it, then run the **whole** suite to find out which tests really catch it. A failure is dropping a
625
+ catching test.
626
+
627
+ | Project | Judged cases | Catching test lost | Suite selected |
628
+ | --- | --- | --- | --- |
629
+ | This repository (node --test, 76 test files) | 8 | **0** | 7–9% |
630
+ | expressjs/express (Mocha, 88 test files) | 2 | **0** | refused to narrow — every test reaches the change |
631
+
632
+ Those two rows are the point. On a layered codebase it runs under a tenth of the suite and keeps every
633
+ test that would have caught the bug; on a monolith where every test imports the package front door it says
634
+ there is nothing to skip, rather than inventing a saving. It is also an MCP tool, so an agent can ask what
635
+ to run before running anything.
636
+
637
+ **In CI**, `mode: pick` runs before your tests and hands the workflow the files to run:
638
+
639
+ ```yaml
640
+ - id: pick
641
+ uses: hmamut39/tewip@main
642
+ with: { mode: pick } # needs fetch-depth: 0 on actions/checkout
643
+ - run: npx jest ${{ steps.pick.outputs.tests }}
644
+ if: steps.pick.outputs.narrowed == 'true'
645
+ - run: npx jest
646
+ if: steps.pick.outputs.narrowed != 'true'
647
+ ```
648
+
649
+ Whatever it leaves out is written into the job summary with the reason beside it, so "why did CI not run
650
+ that test" is answered by the run itself. When it will not narrow safely, `narrowed` is `false` and
651
+ `tests` is empty - so a workflow that ignores the flag still runs everything, which is the safe default.
652
+
555
653
  ## Would your tests catch a bug? (Phase 23)
556
654
 
557
655
  ```bash
package/dist/cli/help.js CHANGED
@@ -118,6 +118,22 @@ Usage:
118
118
  is what was asked for covered. Answers ready, not ready, or cannot
119
119
  say - and cannot say is never rounded up to ready. --json writes a
120
120
  sealed verdict that can be checked later. No model, no cost.
121
+ tewip schema [<files>] [--diff main...HEAD] [--strict] [--out schema.md]
122
+ What will this migration do to a live database? Names the changes
123
+ that break the version still running during a deploy, destroy data,
124
+ block writes while they run, or cannot be undone - and searches your
125
+ code for what still uses the column it takes away. Reads SQL, Django,
126
+ Rails and Alembic migrations, and says so when it cannot read one.
127
+ No model, no cost.
128
+ tewip pick [--diff main...HEAD] [--list] [--out picked.md]
129
+ Which tests could notice this change? Narrows a test run using
130
+ facts rather than a prediction: a test runs when it imports a
131
+ changed file or reaches one through another module, when the change
132
+ edits the test itself, or when it failed recently - an unknown is
133
+ never a skip. It runs the whole suite when the change touches
134
+ dependencies, build or test configuration, shared setup or
135
+ fixtures, or when a changed file cannot be traced to any test.
136
+ --list prints the files for piping. No model, no cost.
121
137
  tewip prove <file> [--diff main...HEAD] [--limit 6] [--budget 600] [--strict]
122
138
  What would your tests NOT catch? Breaks your code in small,
123
139
  deliberate ways - a boundary moved by one, a condition inverted,
package/dist/cli/tewip.js CHANGED
@@ -199,6 +199,30 @@ async function main() {
199
199
  });
200
200
  return;
201
201
  }
202
+ case 'schema': {
203
+ const { runSchema } = await import('../engine/schema/schema-command.js');
204
+ runSchema({
205
+ root: process.cwd(),
206
+ diff: flag(args, 'diff') ?? 'main...HEAD',
207
+ files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out'),
208
+ out: flag(args, 'out'),
209
+ strict: args.includes('--strict'),
210
+ });
211
+ return;
212
+ }
213
+ case 'pick': {
214
+ const { runPick } = await import('../engine/pick/pick-command.js');
215
+ runPick({
216
+ root: process.cwd(),
217
+ diff: flag(args, 'diff') ?? 'main...HEAD',
218
+ files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out' && args[i - 1] !== '--recent' && args[i - 1] !== '--name'),
219
+ out: flag(args, 'out'),
220
+ list: args.includes('--list'),
221
+ recentRuns: Number(flag(args, 'recent') ?? '5'),
222
+ name: flag(args, 'name'),
223
+ });
224
+ return;
225
+ }
202
226
  case 'prove': {
203
227
  const paths = args.filter((a, i) => !a.startsWith('--') && !['--out', '--diff', '--limit', '--budget'].includes(args[i - 1] ?? ''));
204
228
  const { runProve } = await import('../engine/prove/prove-command.js');
@@ -0,0 +1,198 @@
1
+ // `tewip pick --diff main...HEAD`: the tests that could notice a change, and the command that runs them.
2
+ //
3
+ // Everything here is a fact read from the project: the diff from git, the test files on disk, the import
4
+ // graph followed outwards from each changed file, and the failures Tewip has kept. Nothing is predicted.
5
+ import { spawnSync } from 'node:child_process';
6
+ import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
7
+ import { dirname, join, relative, resolve } from 'node:path';
8
+ import { chooseFramework, detectFrameworks } from '../quality/frameworks.js';
9
+ import { changeSummary } from '../quality/grounding.js';
10
+ import { coverageFor, testFiles } from '../review/coverage.js';
11
+ import { importingTests } from '../prove/prove.js';
12
+ import { allTestsReaching } from '../prove/reaches.js';
13
+ import { datasetFiles, readJsonl, readTriageRecords } from '../../reporters/store.js';
14
+ import { pick, renderPick } from './pick.js';
15
+ const slash = (p) => p.split(String.fromCharCode(92)).join('/');
16
+ /** Tests that failed in the most recent runs, by the file they live in. */
17
+ function recentFailures(root, only, howMany) {
18
+ const dir = join(root, '.tewip', 'history');
19
+ const byFile = new Map();
20
+ if (!existsSync(dir))
21
+ return { byFile, noHistory: true };
22
+ const runs = [];
23
+ const records = [];
24
+ for (const project of only ? [only] : readdirSync(dir)) {
25
+ const where = join(dir, project);
26
+ if (!existsSync(where))
27
+ continue;
28
+ runs.push(...readJsonl(datasetFiles(where).runs));
29
+ records.push(...readTriageRecords(datasetFiles(where).triage));
30
+ }
31
+ if (runs.length === 0)
32
+ return { byFile, noHistory: true };
33
+ const recent = new Set(runs
34
+ .slice()
35
+ .sort((a, b) => (a.finishedAt > b.finishedAt ? -1 : 1))
36
+ .slice(0, howMany)
37
+ .map((r) => r.runId));
38
+ for (const record of records) {
39
+ if (!recent.has(record.runId))
40
+ continue;
41
+ const file = slash(String(record.file ?? ''));
42
+ if (!file)
43
+ continue;
44
+ byFile.set(file, [...(byFile.get(file) ?? []), record.testId]);
45
+ }
46
+ return { byFile, noHistory: false };
47
+ }
48
+ /** A test file's path as the history records it, which may be a bare name rather than a full path. */
49
+ const sameTest = (testFile, recorded) => testFile === recorded || testFile.endsWith(`/${recorded}`) || recorded.endsWith(`/${testFile}`);
50
+ /**
51
+ * Does this test walk the repository at run time?
52
+ *
53
+ * Such a test - a guard that scans every source file, a check on licence headers, a snapshot of the
54
+ * public API - can be broken by any file in the project, so it must never be skipped. This is not a
55
+ * guess at coverage: it is a statement about what the test reads. Found the hard way, by a safety
56
+ * measurement that caught this project's own control-character guard failing over a change to a module
57
+ * it does not import.
58
+ */
59
+ function readsTheRepo(root, testFile) {
60
+ try {
61
+ const text = readFileSync(resolve(root, testFile), 'utf8');
62
+ const walks = /\breaddirSync\b|\bglob(?:Sync)?\s*\(|\bwalk\s*\(|os\.walk|Files\.walk|filepath\.Walk/.test(text);
63
+ const reads = /\breadFileSync\b|\bopen\s*\(|readFile\b/.test(text);
64
+ return walks && reads;
65
+ }
66
+ catch {
67
+ return false;
68
+ }
69
+ }
70
+ /**
71
+ * One command rather than a chain of them. Runners take several files at once, and a line of forty
72
+ * `&&`-joined invocations pays the start-up cost forty times - which is most of the saving the selection
73
+ * was for. The shared prefix is taken from the framework's own command for two files, so nothing is
74
+ * assumed about how it is spelled.
75
+ */
76
+ function oneCommand(root, picked, framework) {
77
+ if (!framework)
78
+ return 'your test command';
79
+ if (picked.mustRun.length === 0)
80
+ return framework.suite;
81
+ const paths = picked.mustRun.map((m) => resolve(root, m.testFile));
82
+ const first = framework.command(paths[0]);
83
+ if (paths.length === 1)
84
+ return withCwd(root, framework.cwd, first);
85
+ // Everything before the first quoted argument is the runner. A longest-common-prefix of two commands
86
+ // looked cleverer and cut inside the first file name, producing `node --test "test/ "test/a.ts"`.
87
+ const quote = first.indexOf('"');
88
+ const prefix = quote > 0 ? first.slice(0, quote).trimEnd() : '';
89
+ if (!prefix)
90
+ return withCwd(root, framework.cwd, picked.mustRun.map((m) => framework.command(resolve(root, m.testFile))).join(' && '));
91
+ const quoted = paths.map((p) => JSON.stringify(relative(framework.cwd, p).split(String.fromCharCode(92)).join('/'))).join(' ');
92
+ return withCwd(root, framework.cwd, `${prefix} ${quoted}`);
93
+ }
94
+ /** Says where to run it, when that is not the project root. */
95
+ const withCwd = (root, cwd, command) => (resolve(cwd) === resolve(root) ? command : `cd ${JSON.stringify(relative(root, cwd).split(String.fromCharCode(92)).join('/'))} && ${command}`);
96
+ /** Test files that mention this file by path or by name, which is weaker than an import but not nothing. */
97
+ function mentionsOf(root, file, tests) {
98
+ // The name **with its extension**, or the path. Matching the bare stem was useless: for `cli/tewip.ts`
99
+ // the stem is "tewip", which appears in nearly every file of this project, so 28 test files were
100
+ // selected on the grounds that they had mentioned the product's own name.
101
+ const named = file.split('/').pop() ?? '';
102
+ if (named.length < 5)
103
+ return [];
104
+ const out = [];
105
+ for (const test of tests) {
106
+ try {
107
+ const text = readFileSync(resolve(root, test), 'utf8');
108
+ if (text.includes(file) || text.includes(named))
109
+ out.push(test);
110
+ }
111
+ catch {
112
+ // Unreadable: it cannot be shown to mention anything.
113
+ }
114
+ }
115
+ return out;
116
+ }
117
+ /** Does anything at all in the project refer to this file? Used only to tell "unknown" from "unreachable". */
118
+ function mentionedAnywhere(root, file) {
119
+ const named = file.split('/').pop() ?? '';
120
+ const where = spawnSync('git', ['grep', '-l', '--', named], { cwd: root, encoding: 'utf8', timeout: 60_000 });
121
+ const hits = (where.stdout ?? '')
122
+ .split(String.fromCharCode(10))
123
+ .map((l) => l.trim())
124
+ .filter((l) => l && l.split(String.fromCharCode(92)).join('/') !== file);
125
+ return hits.length > 0;
126
+ }
127
+ export function choose(cmd) {
128
+ const change = cmd.files && cmd.files.length > 0 ? { files: cmd.files } : changeSummary(cmd.root, cmd.diff);
129
+ if (!change)
130
+ return null;
131
+ const changed = change.files.map(slash);
132
+ const known = testFiles(cmd.root).map(slash);
133
+ const relative = known.map((t) => slash(t).replace(`${slash(cmd.root)}/`, ''));
134
+ const { byFile, noHistory } = recentFailures(cmd.root, cmd.name, cmd.recentRuns ?? 5);
135
+ const candidates = relative.map((testFile) => {
136
+ const failed = [];
137
+ for (const [file, ids] of byFile)
138
+ if (sameTest(testFile, file))
139
+ failed.push(...ids);
140
+ return { testFile, recentlyFailed: failed, readsTheRepo: readsTheRepo(cmd.root, testFile) };
141
+ });
142
+ // The import graph, followed outwards from each changed source file.
143
+ const reaching = new Map();
144
+ const sources = changed.filter((f) => existsSync(resolve(cmd.root, f)));
145
+ for (const file of sources) {
146
+ // Both kinds, together: a test may import the file itself and others reach it through the package.
147
+ // Taking only the direct importers when there are any is how a selection quietly drops the tests
148
+ // that exercise the code through the application's own front door.
149
+ const direct = importingTests(cmd.root, coverageFor(cmd.root, [file], testFiles(cmd.root)), file);
150
+ const reached = allTestsReaching(cmd.root, file, relative);
151
+ const byFile = new Map();
152
+ for (const r of reached)
153
+ byFile.set(slash(r.testFile), { testFile: slash(r.testFile), how: `reaches, through ${r.steps} import(s),` });
154
+ for (const testFile of direct)
155
+ byFile.set(slash(testFile), { testFile: slash(testFile), how: 'imports' });
156
+ reaching.set(file, [...byFile.values()]);
157
+ }
158
+ // A file nothing imports may still be named somewhere - a test that reads it as text, a path in a
159
+ // script, a dynamic import. A mention is not proof that a test exercises it, so a test that mentions
160
+ // it runs; and when nothing in the project mentions it at all, nothing can notice it.
161
+ const unreferenced = [];
162
+ for (const [file, reaches] of reaching) {
163
+ if (reaches.length > 0)
164
+ continue;
165
+ const named = mentionsOf(cmd.root, file, relative);
166
+ if (named.length > 0)
167
+ reaching.set(file, named.map((testFile) => ({ testFile, how: 'names' })));
168
+ else if (!mentionedAnywhere(cmd.root, file))
169
+ unreferenced.push(file);
170
+ }
171
+ const framework = chooseFramework(detectFrameworks(cmd.root), undefined, false) ?? detectFrameworks(cmd.root)[0] ?? null;
172
+ const picked = pick({ changed, tests: candidates, reaching, browserTests: Boolean(framework?.browser), noHistory, unreferenced });
173
+ const command = oneCommand(cmd.root, picked, framework);
174
+ return { picked, total: candidates.length, command, noHistory };
175
+ }
176
+ export function runPick(cmd) {
177
+ const chosen = choose(cmd);
178
+ if (!chosen)
179
+ throw new Error(`could not read the change "${cmd.diff}". Is this a git checkout, and does that range exist? (try --diff main...HEAD)`);
180
+ if (cmd.list) {
181
+ // For piping into a runner: the files, and nothing else. A run-everything decision prints nothing,
182
+ // so a script that pipes this cannot mistake "cannot narrow" for "no tests to run".
183
+ if (chosen.picked.runEverything) {
184
+ console.error(`tewip pick: ${chosen.picked.runEverything}. Run the whole suite.`);
185
+ process.exitCode = 2;
186
+ return;
187
+ }
188
+ for (const must of chosen.picked.mustRun)
189
+ console.log(must.testFile);
190
+ return;
191
+ }
192
+ const report = renderPick(chosen.picked, chosen.total, chosen.command, chosen.noHistory);
193
+ console.log(report);
194
+ if (cmd.out) {
195
+ mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
196
+ writeFileSync(resolve(cmd.out), report + String.fromCharCode(10), 'utf8');
197
+ }
198
+ }
@@ -0,0 +1,117 @@
1
+ /** A changed file that can affect anything, so no test can be ruled out because of it. */
2
+ const GLOBAL = [
3
+ { match: /(^|\/)(package|package-lock|yarn|pnpm-lock|bun\.lock|requirements|pyproject|poetry\.lock|go\.mod|go\.sum|pom\.xml|build\.gradle|Gemfile|composer\.json|Cargo\.toml)/i, why: 'it changes the dependencies, which every test runs against' },
4
+ { match: /(^|\/)(\.github|\.gitlab-ci|Jenkinsfile|azure-pipelines|\.circleci|Dockerfile|docker-compose|Makefile)/i, why: 'it changes how the suite is built or run' },
5
+ { match: /(^|\/)(jest|vitest|playwright|cypress|karma|babel|tsconfig|webpack|vite|rollup|eslint|tox|pytest|conftest)\b[\w.]*\.(json|js|ts|cjs|mjs|ini|cfg|toml|yaml|yml)$/i, why: 'it is test or build configuration, which applies to the whole suite' },
6
+ { match: /(^|\/)(conftest\.py|setup\.(js|ts|py)|global-setup\.[jt]s|jest\.setup\.[jt]s|test-setup\.[jt]s)$/i, why: 'it is shared setup, which runs before every test' },
7
+ { match: /(^|\/)(fixtures?|helpers?|support|mocks?|__mocks__|testdata|factories)\//i, why: 'it is shared test support, which any test may use' },
8
+ ];
9
+ const SOURCE = /\.(js|jsx|mjs|cjs|ts|tsx|mts|cts|py|java|kt|go|cs|rb|php|vue|svelte|scala|swift)$/i;
10
+ const IS_TEST = /(^|\/)(tests?|spec|specs|__tests__|e2e)\/|[._-](test|tests|spec|specs)\.[a-z]+$|(^|\/)test_[^/]+\.py$|_test\.(go|py|rb)$|Tests?\.(java|kt|cs)$/i;
11
+ /** Documentation and assets: a test cannot notice these, so they never widen the run. */
12
+ const INERT = /\.(md|mdx|txt|rst|png|jpe?g|gif|svg|ico|webp|woff2?|ttf|eot|mp4|pdf|lock|csv)$|(^|\/)(LICENSE|CODEOWNERS|\.gitignore|\.editorconfig|\.prettierrc)/i;
13
+ export function pick(input) {
14
+ // Global first, then inert. The other order called `requirements.txt` documentation because of its
15
+ // extension and then reported that a dependency change could not affect any test.
16
+ const changed = input.changed.filter((f) => GLOBAL.some((g) => g.match.test(f)) || !INERT.test(f));
17
+ const picked = { mustRun: [], skipped: [], runEverything: '', opaque: [] };
18
+ if (changed.length === 0) {
19
+ return { ...picked, runEverything: 'nothing in this change can affect a test result - only documentation, assets or lockless files were touched. Running nothing is also an option.' };
20
+ }
21
+ // Anything global stops selection outright, and names the file that did it.
22
+ for (const file of changed) {
23
+ const global = GLOBAL.find((g) => g.match.test(file));
24
+ if (global)
25
+ return { ...picked, runEverything: `${file} cannot be narrowed around: ${global.why}` };
26
+ }
27
+ const changedSource = changed.filter((f) => SOURCE.test(f) && !IS_TEST.test(f));
28
+ const changedTests = changed.filter((f) => IS_TEST.test(f));
29
+ // A changed file nothing can be traced from. For a browser suite this is the normal case rather than a
30
+ // gap - the test reaches the code through the running application - so the honest answer is to run
31
+ // everything and say why, not to claim the tests are unrelated.
32
+ // A file no test imports may still be reachable in a way imports do not show. But a file that nothing
33
+ // in the project refers to **at all** - not imported, not named in any source or test - cannot be
34
+ // noticed by any test, and treating that as "cannot tell" made the command useless: in this very
35
+ // repository a change to the command-line dispatcher, which tests only read as text, forced the whole
36
+ // suite. So the two cases are separated, and only a genuine unknown stops selection.
37
+ picked.opaque = changedSource.filter((f) => (input.reaching.get(f) ?? []).length === 0 && !input.unreferenced.includes(f));
38
+ if (picked.opaque.length > 0) {
39
+ return {
40
+ ...picked,
41
+ runEverything: input.browserTests
42
+ ? `these tests drive a browser, so what a test exercises is not written in its imports: ${picked.opaque.slice(0, 3).join(', ')} cannot be traced to any test, and skipping on that basis would be a guess`
43
+ : `no test can be traced to ${picked.opaque.slice(0, 3).join(', ')}${picked.opaque.length > 3 ? ` and ${picked.opaque.length - 3} more` : ''}. Tewip will not take that as proof the suite is unaffected - code can be reached in ways an import does not show (dependency injection, a plugin registry, a dynamic import)`,
44
+ };
45
+ }
46
+ const why = new Map();
47
+ const add = (testFile, reason) => {
48
+ if (!why.has(testFile))
49
+ why.set(testFile, reason);
50
+ };
51
+ // A test the change itself edited always runs: whatever it now checks, nobody has seen it run yet.
52
+ for (const file of changedTests)
53
+ add(file, 'this change edits it');
54
+ for (const [file, reaches] of input.reaching) {
55
+ if (!changedSource.includes(file))
56
+ continue;
57
+ for (const reach of reaches)
58
+ add(reach.testFile, `it ${reach.how} ${file}`);
59
+ }
60
+ // A test that walks the repository can be broken by any file in it, so it always runs.
61
+ for (const test of input.tests) {
62
+ if (test.readsTheRepo)
63
+ add(test.testFile, 'it reads the repository itself, so any changed file can break it');
64
+ }
65
+ // An unknown is not a skip. A test that has been failing or flaking lately gets another run, because
66
+ // "it is unrelated to this change" and "it is broken for another reason" are different statements and
67
+ // only the first is a reason to skip.
68
+ for (const test of input.tests) {
69
+ if (test.recentlyFailed.length > 0)
70
+ add(test.testFile, `it failed recently (${test.recentlyFailed.length} time(s)), so its state is unknown rather than unrelated`);
71
+ }
72
+ picked.mustRun = [...why.entries()].map(([testFile, reason]) => ({ testFile, why: reason })).sort((a, b) => a.testFile.localeCompare(b.testFile));
73
+ picked.skipped = input.tests
74
+ .filter((t) => !why.has(t.testFile))
75
+ .map((t) => ({ testFile: t.testFile, why: 'nothing it imports, directly or through another module, reaches anything this change touched' }))
76
+ .sort((a, b) => a.testFile.localeCompare(b.testFile));
77
+ if (picked.mustRun.length === 0) {
78
+ return {
79
+ ...picked,
80
+ runEverything: `nothing in this project refers to ${changed.slice(0, 3).join(', ')}, so no test can notice the change. Running nothing is defensible; running the suite costs nothing but time.`,
81
+ };
82
+ }
83
+ // Selecting everything is not a failure, but it should be said plainly rather than dressed up as a saving.
84
+ if (picked.skipped.length === 0 && picked.mustRun.length > 0) {
85
+ picked.runEverything = 'every test in the project reaches something this change touched, so there is nothing to skip';
86
+ }
87
+ if (input.noHistory && picked.skipped.length > 0) {
88
+ picked.runEverything = picked.runEverything || '';
89
+ }
90
+ return picked;
91
+ }
92
+ /** What the selection is worth, and what it rests on. */
93
+ export function renderPick(picked, total, command, noHistory) {
94
+ const lines = ['# Which tests could notice this change?', ''];
95
+ if (picked.runEverything) {
96
+ lines.push(`**Run the whole suite.** ${picked.runEverything}.`, '', 'Tewip narrows a test run only when it can show that the tests it leaves out could not have noticed', 'the change. It cannot show that here, so it is not asking you to take the risk.', '');
97
+ if (picked.opaque.length > 0) {
98
+ lines.push('Changed files nothing could be traced from:', '', ...picked.opaque.slice(0, 10).map((f) => `- ${f}`), '');
99
+ }
100
+ return lines.concat(tail(noHistory)).join(String.fromCharCode(10));
101
+ }
102
+ const kept = picked.mustRun.length;
103
+ const share = total > 0 ? Math.round((kept / total) * 100) : 100;
104
+ lines.push(`**${kept} of ${total} test file(s)** could notice it - ${share}% of the suite. ${picked.skipped.length} can be left out.`, '', '```', command, '```', '', `## Why these ${kept} run`, '', ...picked.mustRun.slice(0, 30).map((m) => `- \`${m.testFile}\` — ${m.why}`), ...(picked.mustRun.length > 30 ? [`- …and ${picked.mustRun.length - 30} more`] : []), '', `## What you are giving up (${picked.skipped.length})`, '', 'Each of these is left out because of a fact about the source, not a prediction:', '', ...picked.skipped.slice(0, 15).map((s) => `- \`${s.testFile}\``), ...(picked.skipped.length > 15 ? [`- …and ${picked.skipped.length - 15} more`] : []), '');
105
+ return lines.concat(tail(noHistory)).join(String.fromCharCode(10));
106
+ }
107
+ const tail = (noHistory) => [
108
+ '---',
109
+ '',
110
+ 'How this decides: a test runs when it imports a changed file, or reaches one through another module,',
111
+ 'or when the change edits the test itself, or when it has failed recently - an unknown is never a skip.',
112
+ 'It runs everything when the change touches dependencies, build or test configuration, shared setup or',
113
+ 'shared fixtures, or when a changed file cannot be traced to any test at all. No model is called.',
114
+ ...(noHistory
115
+ ? ['', 'This project keeps no run history, so "it has not failed lately" could not be checked for any test.', 'Run `tewip explain` on your CI reports and the next selection will know more.']
116
+ : []),
117
+ ];
@@ -24,10 +24,14 @@ const NOT_BEHAVIOUR = /^\s*(\/\/|#|\*|\/\*|@|import\b|from\b|export\s+(type|inte
24
24
  * sign the wrong way round, a flag returned the wrong way.
25
25
  */
26
26
  const OPERATORS = [
27
- { kind: 'a boundary moved by one', find: /(?<![<>=!])<=(?!=)/, swap: () => '<' },
28
- { kind: 'a boundary moved by one', find: /(?<![<>=!])>=(?!=)/, swap: () => '>' },
29
- { kind: 'a boundary moved by one', find: /(?<![<>=!+-])<(?![<=])/, swap: () => '<=' },
30
- { kind: 'a boundary moved by one', find: /(?<![<>=!+-])>(?![>=])/, swap: () => '>=' },
27
+ // Whitespace on both sides, which is what separates a comparison from a generic type. Without it,
28
+ // `Array<{ id: string }>` was read as a comparison and mutated into `Array<={ … }>`: the file stopped
29
+ // compiling, the whole suite failed to load, and the measurement counted a mutant nothing caught.
30
+ // A codebase that writes `a<b` loses a few mutants; one that breaks its build loses the answer.
31
+ { kind: 'a boundary moved by one', find: /(?<=\s)<=(?=\s)/, swap: () => '<' },
32
+ { kind: 'a boundary moved by one', find: /(?<=\s)>=(?=\s)/, swap: () => '>' },
33
+ { kind: 'a boundary moved by one', find: /(?<=\s)<(?=\s)/, swap: () => '<=' },
34
+ { kind: 'a boundary moved by one', find: /(?<=\s)>(?=\s)/, swap: () => '>=' },
31
35
  { kind: 'a comparison inverted', find: /===/, swap: () => '!==' },
32
36
  { kind: 'a comparison inverted', find: /(?<![!=<>])==(?!=)/, swap: () => '!=' },
33
37
  { kind: 'a comparison inverted', find: /!==/, swap: () => '===' },
@@ -80,10 +80,11 @@ export function proveFile(root, file, covering, framework, opts = {}) {
80
80
  return finish(proof, started);
81
81
  }
82
82
  const original = readFileSync(full, 'utf8');
83
+ const stopGuarding = guardAgainstBeingKilled(full, original);
83
84
  const mutants = mutantsFor(original, limit);
84
85
  if (mutants.length === 0) {
85
86
  proof.blocked = 'nothing in this file could be changed in a way a test should be expected to catch - no comparison, no condition, no returned value outside of strings and comments';
86
- return finish(proof, started);
87
+ return finish(proof, started, stopGuarding);
87
88
  }
88
89
  // A test has to pass before it can be asked to notice anything, or a red suite turns into "your
89
90
  // tests catch nothing", which is the worst kind of lie this command could tell.
@@ -103,7 +104,7 @@ export function proveFile(root, file, covering, framework, opts = {}) {
103
104
  }
104
105
  if (usable.length === 0) {
105
106
  proof.blocked = `none of the ${covering.length} test file(s) covering this pass as they are, so none can be asked whether they would notice a change. Fix the run first.`;
106
- return finish(proof, started);
107
+ return finish(proof, started, stopGuarding);
107
108
  }
108
109
  proof.covering = usable;
109
110
  // The control: break the file outright and check that a covering test notices. Without this, a file
@@ -111,7 +112,7 @@ export function proveFile(root, file, covering, framework, opts = {}) {
111
112
  say('checking the tests really load this file');
112
113
  if (!testsLoadIt(root, full, original, usable[0], framework, runMs)) {
113
114
  proof.blocked = `${usable[0]} passes even when this file is made unusable, so the tests are not loading the file that was edited - they are reading an installed copy or a build output. Nothing here can be proved until they run against this source (for a Python src layout, install it in editable mode: pip install -e .).`;
114
- return finish(proof, started);
115
+ return finish(proof, started, stopGuarding);
115
116
  }
116
117
  // Only the tests that pass may judge a mutant. A test that is already failing fails for every
117
118
  // change too, which would have it "catch" everything - the exact false confidence this command
@@ -156,7 +157,7 @@ export function proveFile(root, file, covering, framework, opts = {}) {
156
157
  }
157
158
  proof.tried.push({ mutant, fate, noticedBy, seconds: Math.round((Date.now() - at) / 100) / 10 });
158
159
  }
159
- return finish(proof, started);
160
+ return finish(proof, started, stopGuarding);
160
161
  }
161
162
  /**
162
163
  * Puts the file back and checks that it really went back. Anything less would mean this command can
@@ -168,7 +169,43 @@ function restore(full, original) {
168
169
  throw new Error(`${full} could not be put back as it was. Check it before doing anything else: git checkout -- "${full}"`);
169
170
  }
170
171
  }
171
- const finish = (proof, started) => ({ ...proof, seconds: Math.round((Date.now() - started) / 1000) });
172
+ /**
173
+ * Put the file back even when the process is killed.
174
+ *
175
+ * `finally` does not run for SIGINT or SIGTERM, so a developer pressing Ctrl-C during a run - which they
176
+ * will, because this is the slow command - was left with a mutated source file and no idea why their
177
+ * code had stopped working. Found by killing one of these runs during development and finding a flipped
178
+ * comparison still sitting in the repository.
179
+ */
180
+ function guardAgainstBeingKilled(full, original) {
181
+ const putBack = (signal) => () => {
182
+ try {
183
+ restore(full, original);
184
+ process.stderr.write(`${String.fromCharCode(10)}${signal}: ${full} was put back as it was.${String.fromCharCode(10)}`);
185
+ }
186
+ catch {
187
+ process.stderr.write(`${String.fromCharCode(10)}${signal}: could not put ${full} back. Run: git checkout -- "${full}"${String.fromCharCode(10)}`);
188
+ }
189
+ process.exit(130);
190
+ };
191
+ const handlers = ['SIGINT', 'SIGTERM', 'SIGHUP', 'SIGBREAK'].map((signal) => [signal, putBack(signal)]);
192
+ for (const [signal, handler] of handlers) {
193
+ try {
194
+ process.on(signal, handler);
195
+ }
196
+ catch {
197
+ // Not every signal exists on every platform.
198
+ }
199
+ }
200
+ return () => {
201
+ for (const [signal, handler] of handlers)
202
+ process.off(signal, handler);
203
+ };
204
+ }
205
+ const finish = (proof, started, stopGuarding) => {
206
+ stopGuarding?.();
207
+ return { ...proof, seconds: Math.round((Date.now() - started) / 1000) };
208
+ };
172
209
  const firstLine = (output) => output.split(String.fromCharCode(10)).map((l) => l.trim()).find(Boolean)?.slice(0, 160) ?? '';
173
210
  /** Which of the tests that cover a file really import it, and so could notice a change. */
174
211
  export function importingTests(root, coverage, file) {