tewip-ai 0.27.0 → 0.28.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -78,6 +78,13 @@ On every run, the action:
78
78
  when one of them cannot fail. Needs `fetch-depth: 0` on `actions/checkout` so the diff can be read;
79
79
  without it the section is skipped rather than guessed at. Any such test also enters the attestation
80
80
  as a refusal, so a release decision taken from the bundle cannot rest on a test that proves nothing.
81
+ - **What the change would slip past (`prove: true`):** breaks the source files the pull request touched
82
+ in small deliberate ways, runs the tests that cover them, and reports every change none of them
83
+ noticed - "these 2 changes to your code would have slipped past your tests", with the lines. Off by
84
+ default because each change costs a real test run; `prove-budget` (default 300s) caps the whole
85
+ thing, and it says which files it did not reach rather than implying they were clean. Every file is
86
+ put back exactly as it was. Each unnoticed change also enters the attestation as a refusal: a line
87
+ nothing checks is not a line this bundle can claim was verified.
81
88
  - **Evidence, every run:** the Action writes an attestation into its artifact - the claims with
82
89
  their evidence and which layer decided each, what Tewip **refused** to decide, and what it did
83
90
  not check at all. Nothing enters it that Tewip did not verify itself, so a release decision can
@@ -503,6 +510,48 @@ Joins the test plans Tewip wrote to the tests that cover them and the runs that
503
510
  scenario counts as covered only when a real test can be named, and passing only when a real run
504
511
  attempted it; everything else says what is missing. No model, no cost.
505
512
 
513
+ ## Is this work actually done? (Phase 24)
514
+
515
+ ```bash
516
+ tewip done --diff main...HEAD # one verdict
517
+ tewip done --prove --strict # also break the code, and fail the build unless ready
518
+ tewip done --json done.json # a sealed verdict that can be checked later
519
+ ```
520
+
521
+ Tewip can answer six questions about a change. A team reviewing an agent's pull request has to run six
522
+ commands and assemble the answer themselves - which in practice means they run none and merge on a
523
+ green tick. `done` is the assembled answer:
524
+
525
+ ```
526
+ # Is this work done? Not ready
527
+
528
+ | Question | Answer |
529
+ | Is anything failing? | not checked |
530
+ | Is the changed code tested? | yes |
531
+ | Can the tests in this change fail? | no |
532
+ | Would the tests catch a bug in this code? | no |
533
+ | Is what was asked for covered? | not checked |
534
+ ```
535
+
536
+ That is a real run on a change where **all three tests passed**. One of them was `assert.ok(true)`, and
537
+ three deliberate changes to the new code went unnoticed by the suite. CI was green; the work was not
538
+ done.
539
+
540
+ Nothing is judged here. Every answer comes from a layer with its own measured accuracy — the triage
541
+ rules, `vouch`, `prove`, `trace` — and this only puts them in a fixed, published order: what is broken,
542
+ then what cannot be believed, then what is missing. The same change always gets the same verdict.
543
+
544
+ **"Cannot say" is a real answer, and it is never rounded up to ready.** A verdict needs the two questions
545
+ nothing can substitute for — is anything failing, and does anything actually test this code — to have
546
+ been answered. If Tewip has never seen a test result for the project, it says so and refuses to approve;
547
+ the first version of this answered "Ready, with 3 not checked", which is an approval assembled out of
548
+ shrugs. Every verdict also ends with what it says nothing about, including the one thing no tool here
549
+ can know: whether the change does what was intended.
550
+
551
+ With `--json` the verdict is **sealed**: editing `not-ready` to `ready` in the file makes verification
552
+ fail, so a verdict that travels into a ticket or a release note can be checked rather than trusted. It is
553
+ also an MCP tool, so an agent can ask whether its own work is done before saying that it is.
554
+
506
555
  ## Would your tests catch a bug? (Phase 23)
507
556
 
508
557
  ```bash
package/dist/cli/help.js CHANGED
@@ -111,6 +111,13 @@ Usage:
111
111
  or attesting a green goal - by running the command that already
112
112
  proves them. It will not touch a real bug, and it writes what it
113
113
  did or refused to .tewip/actions.jsonl.
114
+ tewip done [--diff main...HEAD] [--prove] [--strict] [--out done.md] [--json done.json]
115
+ Is this work actually done? One verdict, from the layers that each
116
+ have a measured accuracy: is anything failing, is the changed code
117
+ tested, can those tests fail, would they catch a bug (--prove), and
118
+ is what was asked for covered. Answers ready, not ready, or cannot
119
+ say - and cannot say is never rounded up to ready. --json writes a
120
+ sealed verdict that can be checked later. No model, no cost.
114
121
  tewip prove <file> [--diff main...HEAD] [--limit 6] [--budget 600] [--strict]
115
122
  What would your tests NOT catch? Breaks your code in small,
116
123
  deliberate ways - a boundary moved by one, a condition inverted,
package/dist/cli/tewip.js CHANGED
@@ -185,6 +185,20 @@ async function main() {
185
185
  runPursue({ root: process.cwd(), goal, act: args.includes('--act'), name: flag(args, 'name'), out: flag(args, 'out') });
186
186
  return;
187
187
  }
188
+ case 'done': {
189
+ const { runDone } = await import('../engine/verdict/verdict-command.js');
190
+ runDone({
191
+ root: process.cwd(),
192
+ diff: flag(args, 'diff') ?? 'main...HEAD',
193
+ prove: args.includes('--prove'),
194
+ proveBudgetSeconds: Number(flag(args, 'budget') ?? '300'),
195
+ out: flag(args, 'out'),
196
+ json: flag(args, 'json'),
197
+ name: flag(args, 'name'),
198
+ strict: args.includes('--strict'),
199
+ });
200
+ return;
201
+ }
188
202
  case 'prove': {
189
203
  const paths = args.filter((a, i) => !a.startsWith('--') && !['--out', '--diff', '--limit', '--budget'].includes(args[i - 1] ?? ''));
190
204
  const { runProve } = await import('../engine/prove/prove-command.js');
@@ -0,0 +1,77 @@
1
+ // Proving what a pull request's own changes would slip past, inside CI.
2
+ //
3
+ // `tewip prove` on a whole file is a thing somebody chooses to wait for. In CI nobody chooses, so this
4
+ // is deliberately meaner: only source files the diff touched, a small number of them, a hard time
5
+ // budget, and off unless the repository asks for it. A tool that silently adds six minutes to every
6
+ // pull request is a tool that gets removed from the workflow, and then it checks nothing at all.
7
+ //
8
+ // It says nothing when nothing slipped past. A section that appears on every pull request to report
9
+ // "all good" is one people stop reading, and then it is worth nothing on the day it matters.
10
+ import { existsSync } from 'node:fs';
11
+ import { resolve } from 'node:path';
12
+ import { chooseFramework, detectFrameworks } from '../quality/frameworks.js';
13
+ import { coverageFor, testFiles } from '../review/coverage.js';
14
+ import { importingTests, proveFile } from './prove.js';
15
+ import { testsReaching } from './reaches.js';
16
+ const SOURCE = /\.(js|jsx|mjs|cjs|ts|tsx|mts|cts|py|java|kt|go|cs|rb|php)$/i;
17
+ const IS_TEST = /(^|[\\/])(tests?|spec|specs|__tests__|e2e)[\\/]|[._-](test|tests|spec|specs)\.[a-z]+$|(^|[\\/])test_[^\\/]+\.py$|_test\.(go|py|rb)$|Tests?\.(java|kt|cs)$/i;
18
+ export function proveChanged(root, changed, budget = {}) {
19
+ const maxFiles = budget.maxFiles ?? 2;
20
+ const totalMs = (budget.budgetSeconds ?? 300) * 1000;
21
+ const say = budget.onStep ?? (() => { });
22
+ const files = changed.map((f) => f.split('\\').join('/')).filter((f) => SOURCE.test(f) && !IS_TEST.test(f) && existsSync(resolve(root, f)));
23
+ if (files.length === 0)
24
+ return { proofs: [], unnoticed: 0, notReached: [], skipped: 'this change touches no source file that could be proved' };
25
+ const framework = chooseFramework(detectFrameworks(root), undefined, false);
26
+ if (!framework)
27
+ return { proofs: [], unnoticed: 0, notReached: [], skipped: 'no unit test framework was found here, and proving works by running tests' };
28
+ const known = testFiles(root);
29
+ const started = Date.now();
30
+ const proofs = [];
31
+ const chosen = files.slice(0, maxFiles);
32
+ const notReached = files.slice(maxFiles);
33
+ for (const file of chosen) {
34
+ const left = totalMs - (Date.now() - started);
35
+ // Under half a minute is not enough for a baseline, a control and one change, and a run cut off
36
+ // half way would report survivors that were never really tried.
37
+ if (left < 30_000) {
38
+ notReached.push(file);
39
+ continue;
40
+ }
41
+ const direct = importingTests(root, coverageFor(root, [file], known), file);
42
+ const covering = direct.length > 0 ? direct : testsReaching(root, file, known.map((t) => t.split('\\').join('/'))).map((r) => r.testFile);
43
+ say(`proving ${file} against ${covering.length} test file(s)`);
44
+ proofs.push(proveFile(root, file, covering, framework, { limit: budget.limit ?? 4, budgetSeconds: Math.floor(left / 1000), onStep: say }));
45
+ }
46
+ return {
47
+ proofs,
48
+ unnoticed: proofs.flatMap((p) => p.tried).filter((t) => t.fate === 'unnoticed').length,
49
+ notReached,
50
+ skipped: '',
51
+ };
52
+ }
53
+ /** The section for the pull request comment. Empty when nothing slipped past. */
54
+ export function provedChangeLines(proved) {
55
+ const gaps = proved.proofs.flatMap((p) => p.tried.filter((t) => t.fate === 'unnoticed').map((t) => ({ file: p.file, tried: t })));
56
+ if (gaps.length === 0)
57
+ return [];
58
+ const lines = [
59
+ `## ${gaps.length} change(s) to this code would have slipped past the tests`,
60
+ '',
61
+ 'Tewip made each of these changes to your own code and ran the tests that cover it. They all still',
62
+ 'passed. The lines are covered; they are not checked.',
63
+ '',
64
+ ];
65
+ for (const gap of gaps.slice(0, 8)) {
66
+ lines.push(`- **\`${gap.file}:${gap.tried.mutant.line}\`** — ${gap.tried.mutant.kind}`);
67
+ lines.push(` - was: \`${gap.tried.mutant.was.slice(0, 110).replace(/`/g, "'")}\``);
68
+ lines.push(` - became: \`${gap.tried.mutant.now.slice(0, 110).replace(/`/g, "'")}\``);
69
+ }
70
+ if (gaps.length > 8)
71
+ lines.push(`- …and ${gaps.length - 8} more`);
72
+ lines.push('', 'Every file was put back exactly as it was. Nothing here is necessarily a bug: it is a place where a mistake would go unnoticed.', '');
73
+ if (proved.notReached.length > 0) {
74
+ lines.push(`Not proved in the time allowed: ${proved.notReached.slice(0, 6).join(', ')}${proved.notReached.length > 6 ? `, and ${proved.notReached.length - 6} more` : ''}.`, '');
75
+ }
76
+ return lines;
77
+ }
@@ -0,0 +1,140 @@
1
+ // `tewip done --diff main...HEAD`: gathering the evidence for the verdict.
2
+ //
3
+ // Every number here comes from a layer that already exists and has its own measured accuracy. This file
4
+ // only collects: what changed, what covers it, whether those tests can fail, whether they would catch a
5
+ // bug, what is failing now, and what a kept plan says is still uncovered. Where a layer has nothing to
6
+ // say, it is left as "not checked" rather than filled in with a guess.
7
+ import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
8
+ import { dirname, join, resolve } from 'node:path';
9
+ import { seal } from '../attest/seal.js';
10
+ import { proveChanged } from '../prove/changed.js';
11
+ import { changeSummary } from '../quality/grounding.js';
12
+ import { coverageFor, testFiles } from '../review/coverage.js';
13
+ import { importingTests } from '../prove/prove.js';
14
+ import { testsReaching } from '../prove/reaches.js';
15
+ import { vouchChanged } from '../vouch/changed.js';
16
+ import { datasetFiles, readJsonl, readTriageRecords } from '../../reporters/store.js';
17
+ import { decide, renderDecision } from './verdict.js';
18
+ const SOURCE = /\.(js|jsx|mjs|cjs|ts|tsx|mts|cts|py|java|kt|go|cs|rb|php)$/i;
19
+ const IS_TEST = /(^|[\\/])(tests?|spec|specs|__tests__|e2e)[\\/]|[._-](test|tests|spec|specs)\.[a-z]+$|(^|[\\/])test_[^\\/]+\.py$|_test\.(go|py|rb)$|Tests?\.(java|kt|cs)$/i;
20
+ /** Every failure Tewip has kept for this project, and every run. */
21
+ function history(root, only) {
22
+ const dir = join(root, '.tewip', 'history');
23
+ if (!existsSync(dir))
24
+ return { runs: [], records: [] };
25
+ const runs = [];
26
+ const records = [];
27
+ for (const project of only ? [only] : readdirSync(dir)) {
28
+ const where = join(dir, project);
29
+ if (!existsSync(where))
30
+ continue;
31
+ runs.push(...readJsonl(datasetFiles(where).runs));
32
+ records.push(...readTriageRecords(datasetFiles(where).triage));
33
+ }
34
+ runs.sort((a, b) => (a.finishedAt > b.finishedAt ? 1 : -1));
35
+ return { runs, records };
36
+ }
37
+ /** Scenarios of kept plans that this change relates to and nothing covers. Null when no plans exist. */
38
+ function uncoveredFor(root, changed, attempted) {
39
+ const dir = join(root, '.tewip', 'plans');
40
+ if (!existsSync(dir))
41
+ return null;
42
+ const words = new Set(changed.flatMap((f) => (f.toLowerCase().match(/[a-z][a-z0-9]{3,}/g) ?? [])));
43
+ const clean = (s) => s.toLowerCase().replace(/[^a-z0-9]+/g, '');
44
+ const out = [];
45
+ for (const file of readdirSync(dir).filter((f) => f.endsWith('.json'))) {
46
+ try {
47
+ const saved = JSON.parse(readFileSync(join(dir, file), 'utf8'));
48
+ const plan = saved.plan ?? saved;
49
+ if (!Array.isArray(plan.scenarios))
50
+ continue;
51
+ // Only plans this change has something to do with: a verdict about an unrelated feature's gaps
52
+ // would block every pull request in the project forever.
53
+ const about = (`${plan.title ?? ''} ${saved.feature ?? ''}`.toLowerCase().match(/[a-z][a-z0-9]{3,}/g) ?? []).some((w) => words.has(w));
54
+ if (!about)
55
+ continue;
56
+ for (const scenario of plan.scenarios) {
57
+ const covered = (scenario.covered_by ?? '').trim() || attempted.find((t) => clean(t).includes(clean(scenario.id)) && clean(scenario.id).length >= 4);
58
+ if (!covered)
59
+ out.push({ id: scenario.id, title: scenario.title });
60
+ }
61
+ }
62
+ catch {
63
+ // Not a readable plan.
64
+ }
65
+ }
66
+ return out;
67
+ }
68
+ export function gather(cmd) {
69
+ const change = changeSummary(cmd.root, cmd.diff);
70
+ if (!change || change.files.length === 0)
71
+ return null;
72
+ const changed = change.files.map((f) => f.split('\\').join('/'));
73
+ const changedSource = changed.filter((f) => SOURCE.test(f) && !IS_TEST.test(f) && existsSync(resolve(cmd.root, f)));
74
+ const changedTests = changed.filter((f) => IS_TEST.test(f) && existsSync(resolve(cmd.root, f)));
75
+ const known = testFiles(cmd.root);
76
+ const covering = [];
77
+ for (const file of changedSource) {
78
+ const direct = importingTests(cmd.root, coverageFor(cmd.root, [file], known), file);
79
+ for (const test of direct)
80
+ if (!covering.some((c) => c.testFile === test))
81
+ covering.push({ testFile: test, how: 'imports it' });
82
+ if (direct.length > 0)
83
+ continue;
84
+ for (const reach of testsReaching(cmd.root, file, known.map((t) => t.split('\\').join('/')))) {
85
+ if (!covering.some((c) => c.testFile === reach.testFile))
86
+ covering.push({ testFile: reach.testFile, how: `reaches it through ${reach.steps} import(s)` });
87
+ }
88
+ }
89
+ const vouched = vouchChanged(cmd.root, changedTests);
90
+ const { runs, records } = history(cmd.root, cmd.name);
91
+ const newest = runs.at(-1);
92
+ const touched = new Set([...changedSource, ...changedTests, ...covering.map((c) => c.testFile)]);
93
+ const failingNow = records
94
+ .filter((r) => r.runId === newest?.runId)
95
+ .filter((r) => {
96
+ const where = String(r.file ?? '').split('\\').join('/');
97
+ return touched.has(where) || [...touched].some((t) => r.testId.includes(t.replace(/\.[a-z]+$/, '')));
98
+ })
99
+ .map((r) => ({ testId: r.testId, category: String(r.diagnosis?.category ?? 'UNKNOWN') }));
100
+ const proved = cmd.prove ? proveChanged(cmd.root, changedSource, { budgetSeconds: cmd.proveBudgetSeconds ?? 300, onStep: (m) => console.error(` ${m}`) }) : null;
101
+ return {
102
+ changed,
103
+ evidence: {
104
+ changedSource,
105
+ changedTests,
106
+ covering,
107
+ cannotFail: vouched.cannotFail.map((j) => ({ name: j.name, file: j.file, line: j.line, why: j.why })),
108
+ crashOnly: vouched.crashOnly.length,
109
+ unnoticed: proved ? proved.proofs.flatMap((p) => p.tried.filter((t) => t.fate === 'unnoticed').map((t) => ({ file: p.file, line: t.mutant.line, kind: t.mutant.kind }))) : null,
110
+ failingNow,
111
+ noHistory: runs.length === 0,
112
+ uncovered: uncoveredFor(cmd.root, changed, Object.keys(newest?.attempts ?? {})),
113
+ },
114
+ };
115
+ }
116
+ export function runDone(cmd) {
117
+ const gathered = gather(cmd);
118
+ if (!gathered)
119
+ throw new Error(`nothing changed in "${cmd.diff}", or that range could not be read. Is this a git checkout? (try --diff main...HEAD)`);
120
+ const decision = decide(gathered.evidence);
121
+ const report = renderDecision(cmd.diff, decision);
122
+ console.log(report);
123
+ if (cmd.out) {
124
+ mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
125
+ writeFileSync(resolve(cmd.out), report + String.fromCharCode(10), 'utf8');
126
+ }
127
+ if (cmd.json) {
128
+ // Sealed, so a verdict that travels - into a ticket, a release note, an audit - can be checked
129
+ // rather than trusted. The digest is of the content; a signature needs TEWIP_ATTEST_KEY.
130
+ const bundle = { range: cmd.diff, changed: gathered.changed, ...decision };
131
+ const sealed = { ...bundle, seal: seal(bundle, process.env.TEWIP_ATTEST_KEY?.includes('BEGIN') ? process.env.TEWIP_ATTEST_KEY : undefined) };
132
+ mkdirSync(dirname(resolve(cmd.json)), { recursive: true });
133
+ writeFileSync(resolve(cmd.json), JSON.stringify(sealed, null, 2), 'utf8');
134
+ }
135
+ if (cmd.strict && decision.verdict !== 'ready') {
136
+ console.error(String.fromCharCode(10) + decision.headline);
137
+ process.exitCode = 1;
138
+ }
139
+ return { verdict: decision.verdict };
140
+ }
@@ -0,0 +1,138 @@
1
+ /** The questions that cannot be skipped on the way to "ready". */
2
+ const ESSENTIAL = ['Is anything failing?', 'Is the changed code tested?'];
3
+ /**
4
+ * The verdict, in a fixed order: what is broken first, then what cannot be believed, then what is
5
+ * missing, and only then "ready".
6
+ */
7
+ export function decide(evidence) {
8
+ const checks = [];
9
+ // 1. Is anything failing right now? Nothing else matters while it is.
10
+ if (evidence.noHistory) {
11
+ checks.push({ question: 'Is anything failing?', standing: 'not-checked', answer: 'no run has been kept for this project, so Tewip has not seen a test result here. Run `tewip explain <report>` in CI and it will know.', decidedBy: 'kept history' });
12
+ }
13
+ else if (evidence.failingNow.length > 0) {
14
+ checks.push({
15
+ question: 'Is anything failing?',
16
+ standing: 'not-met',
17
+ answer: `${evidence.failingNow.length} test(s) touching this change failed in the last run: ${evidence.failingNow.slice(0, 3).map((f) => `${f.testId} (${f.category})`).join(', ')}`,
18
+ decidedBy: 'the triage rules and model, measured on real CI data',
19
+ });
20
+ }
21
+ else {
22
+ checks.push({ question: 'Is anything failing?', standing: 'met', answer: 'nothing touching this change failed in the last kept run', decidedBy: 'kept history' });
23
+ }
24
+ // 2. Does anything test the change at all?
25
+ if (evidence.changedSource.length === 0) {
26
+ checks.push({ question: 'Is the changed code tested?', standing: 'not-applicable', answer: 'this change touches no source file', decidedBy: 'the diff' });
27
+ }
28
+ else if (evidence.covering.length === 0) {
29
+ checks.push({
30
+ question: 'Is the changed code tested?',
31
+ standing: 'not-met',
32
+ answer: `no test imports or reaches ${evidence.changedSource.slice(0, 3).join(', ')}, so nothing here could notice a mistake in it`,
33
+ decidedBy: 'imports followed through the source, which is a fact rather than a name match',
34
+ });
35
+ }
36
+ else {
37
+ checks.push({
38
+ question: 'Is the changed code tested?',
39
+ standing: 'met',
40
+ answer: `${evidence.covering.length} test file(s) reach it: ${evidence.covering.slice(0, 3).map((c) => c.testFile).join(', ')}`,
41
+ decidedBy: 'imports followed through the source',
42
+ });
43
+ }
44
+ // 3. Can the tests in this change fail at all? A test that cannot fail is worse than no test.
45
+ if (evidence.changedTests.length === 0) {
46
+ checks.push({ question: 'Can the tests in this change fail?', standing: 'not-applicable', answer: 'this change adds or edits no test', decidedBy: 'the diff' });
47
+ }
48
+ else if (evidence.cannotFail.length > 0) {
49
+ checks.push({
50
+ question: 'Can the tests in this change fail?',
51
+ standing: 'not-met',
52
+ answer: `${evidence.cannotFail.length} of them cannot: ${evidence.cannotFail.slice(0, 2).map((t) => `${t.name} (${t.file}:${t.line})`).join(', ')}`,
53
+ decidedBy: 'tewip vouch, 19 of 19 correct on 5,448 tests in 8 public projects',
54
+ });
55
+ }
56
+ else {
57
+ checks.push({
58
+ question: 'Can the tests in this change fail?',
59
+ standing: 'met',
60
+ answer: `every test in this change asserts something${evidence.crashOnly > 0 ? `, though ${evidence.crashOnly} compare no value and would only fail on a crash` : ''}`,
61
+ decidedBy: 'tewip vouch',
62
+ });
63
+ }
64
+ // 4. Would they catch a bug? The strongest check, and the one that costs the most, so often skipped.
65
+ if (evidence.unnoticed === null) {
66
+ checks.push({ question: 'Would the tests catch a bug in this code?', standing: 'not-checked', answer: 'not run: this means breaking the code on purpose and running the tests, which costs minutes. Ask for it with `--prove`.', decidedBy: 'tewip prove' });
67
+ }
68
+ else if (evidence.unnoticed.length > 0) {
69
+ checks.push({
70
+ question: 'Would the tests catch a bug in this code?',
71
+ standing: 'not-met',
72
+ answer: `${evidence.unnoticed.length} deliberate change(s) went unnoticed, including ${evidence.unnoticed[0].file}:${evidence.unnoticed[0].line} (${evidence.unnoticed[0].kind})`,
73
+ decidedBy: 'tewip prove, by running the tests against a broken copy of the code',
74
+ });
75
+ }
76
+ else {
77
+ checks.push({ question: 'Would the tests catch a bug in this code?', standing: 'met', answer: 'every deliberate change made to this code was caught by a covering test', decidedBy: 'tewip prove' });
78
+ }
79
+ // 5. Is what was asked for covered? Only where the project keeps plans to compare against.
80
+ if (evidence.uncovered === null) {
81
+ checks.push({ question: 'Is what was asked for covered?', standing: 'not-checked', answer: 'this project keeps no test plan Tewip can read, so there is nothing to compare the change against', decidedBy: 'tewip trace' });
82
+ }
83
+ else if (evidence.uncovered.length > 0) {
84
+ checks.push({
85
+ question: 'Is what was asked for covered?',
86
+ standing: 'not-met',
87
+ answer: `${evidence.uncovered.length} scenario(s) related to this change have no test: ${evidence.uncovered.slice(0, 2).map((u) => `${u.id} ${u.title}`).join('; ')}`,
88
+ decidedBy: 'tewip trace, which counts a scenario covered only when a real test can be named',
89
+ });
90
+ }
91
+ else {
92
+ checks.push({ question: 'Is what was asked for covered?', standing: 'met', answer: 'every scenario this change relates to has a test that can be named', decidedBy: 'tewip trace' });
93
+ }
94
+ const broken = checks.filter((c) => c.standing === 'not-met');
95
+ const unknown = checks.filter((c) => c.standing === 'not-checked');
96
+ // "Ready" requires that the two questions nothing can substitute for were actually answered: is
97
+ // anything failing, and does anything test the changed code. Without those, "ready" would rest on
98
+ // the checks that happened to be cheap - the first version said Ready on a project with no history
99
+ // at all, which is an approval built out of three shrugs.
100
+ const essential = checks.filter((c) => ESSENTIAL.includes(c.question));
101
+ const verdict = broken.length > 0 ? 'not-ready' : essential.some((c) => c.standing === 'not-checked') ? 'cannot-say' : 'ready';
102
+ return {
103
+ verdict,
104
+ headline: broken.length > 0
105
+ ? `Not ready: ${broken[0].answer}`
106
+ : verdict === 'cannot-say'
107
+ ? `Cannot say: ${essential.filter((c) => c.standing === 'not-checked').map((c) => c.question.toLowerCase().replace(/\?$/, '')).join(' and ')} could not be checked, so this is not an approval`
108
+ : `Ready as far as ${checks.filter((c) => c.standing === 'met').length} check(s) can show${unknown.length > 0 ? `, with ${unknown.length} not checked` : ''}`,
109
+ checks,
110
+ notChecked: [
111
+ ...unknown.map((c) => `${c.question.toLowerCase().replace(/\?$/, '')} - ${c.answer}`),
112
+ 'whether the change does what was intended, which no tool here can know',
113
+ 'the running application, its dependencies and its secrets, unless `tewip secure` was run separately',
114
+ ],
115
+ };
116
+ }
117
+ export function renderDecision(range, decision) {
118
+ const mark = { met: 'yes', 'not-met': '**no**', 'not-applicable': 'n/a', 'not-checked': 'not checked' };
119
+ const title = decision.verdict === 'ready' ? 'Ready, as far as this can be shown' : decision.verdict === 'not-ready' ? 'Not ready' : 'Cannot say';
120
+ const lines = [
121
+ `# Is this work done? ${title}`,
122
+ '',
123
+ `For \`${range}\`. ${decision.headline}.`,
124
+ '',
125
+ '| Question | Answer |',
126
+ '| --- | --- |',
127
+ ...decision.checks.map((c) => `| ${c.question} | ${mark[c.standing]} |`),
128
+ '',
129
+ ];
130
+ for (const check of decision.checks) {
131
+ if (check.standing === 'not-applicable')
132
+ continue;
133
+ const said = `${mark[check.standing] === '**no**' ? 'No' : mark[check.standing] === 'yes' ? 'Yes' : 'Not checked'}: ${check.answer}`;
134
+ lines.push(`### ${check.question}`, '', /[.!?]$/.test(said) ? said : `${said}.`, '', `_Decided by ${check.decidedBy}._`, '');
135
+ }
136
+ lines.push('## What this says nothing about', '', ...decision.notChecked.map((n) => `- ${n}`), '', '---', '', 'Nothing above is judged here: each answer comes from a layer with its own measured accuracy, and', 'this only puts them in a fixed order - what is broken, then what cannot be believed, then what is', 'missing. "Cannot say" is a real answer and is never rounded up to ready.');
137
+ return lines.join(String.fromCharCode(10));
138
+ }
@@ -125,6 +125,11 @@ export function tautologies(body) {
125
125
  if (left && right && left === right && !/^(true|false|null|undefined|None)$/.test(left))
126
126
  found.push(line.trim());
127
127
  }
128
+ // `assert.ok(true)`, `assert(true)`, `assertTrue(true)`: a check whose subject is the answer. Found
129
+ // while writing a realistic fixture for the verdict - the commonest placeholder in Node's own test
130
+ // style, and the vocabulary did not have it.
131
+ if (/\b(assert(?:\.(?:ok|isTrue|isOk))?|assertTrue|expect)\s*\(\s*(?:true|1|!0)\s*[,)]/.test(line))
132
+ found.push(line.trim());
128
133
  // `assert True` / `assert 1`: a constant truth written as a statement.
129
134
  if (/^\s*assert\s+(True|true|1)\s*(,|$)/.test(line))
130
135
  found.push(line.trim());
@@ -155,4 +155,25 @@ server.registerTool('prove_tests_catch_bugs', {
155
155
  const { proveOne } = await import('../engine/prove/prove-command.js');
156
156
  return proveOne(root, file, { limit, budgetSeconds });
157
157
  }));
158
+ server.registerTool('is_the_work_done', {
159
+ title: 'Is this work actually done?',
160
+ description: "One verdict on a change, from the layers that each have their own measured accuracy: is anything failing, is the changed code tested by something that really reaches it, can the tests in the change fail at all, would they catch a bug (only when prove is asked for, because that costs minutes), and is what was asked for covered. Answers ready, not ready, or cannot say - and cannot say is never rounded up to ready. Use it before claiming a piece of work is finished, including your own: a green test run is not the same as work that is done.",
161
+ inputSchema: {
162
+ project,
163
+ diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
164
+ prove: z.boolean().optional().describe('Also break the changed code and run the tests against it. Costs minutes.'),
165
+ },
166
+ annotations: readOnly,
167
+ }, ({ project: root, diff, prove }) => answer(async () => {
168
+ if (!isAbsolute(root) || !existsSync(root))
169
+ throw new Error(`project must be an existing absolute path, got "${root}"`);
170
+ // gather/decide/render are pure; runDone prints, which would corrupt the protocol.
171
+ const { gather } = await import('../engine/verdict/verdict-command.js');
172
+ const { decide, renderDecision } = await import('../engine/verdict/verdict.js');
173
+ const range = diff ?? 'main...HEAD';
174
+ const gathered = gather({ root, diff: range, prove });
175
+ if (!gathered)
176
+ return `Nothing changed in ${range}, or that range could not be read here.`;
177
+ return renderDecision(range, decide(gathered.evidence));
178
+ }));
158
179
  await server.connect(new StdioServerTransport());
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "tewip-ai",
3
- "version": "0.27.0",
3
+ "version": "0.28.0",
4
4
  "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
5
5
  "keywords": [
6
6
  "test-failures",