tewip-ai 0.32.0 → 0.34.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -3
- package/dist/cli/tewip.js +4 -69
- package/dist/cli/verbs-prove.js +82 -0
- package/dist/engine/import/case.js +16 -0
- package/dist/engine/import/dotnet.js +58 -0
- package/dist/engine/import/junit.js +2 -64
- package/dist/mcp/server.js +18 -113
- package/dist/mcp/tools-prove.js +116 -0
- package/package.json +82 -81
package/README.md
CHANGED
|
@@ -1,6 +1,10 @@
|
|
|
1
1
|
# Tewip
|
|
2
2
|
|
|
3
|
-
**Tewip** (Uyghur *téwip*, تېۋىپ: healer, doctor)
|
|
3
|
+
**Tewip** (Uyghur *téwip*, تېۋىپ: healer, doctor) is the layer that **proves software work** - written by a person or by an agent.
|
|
4
|
+
|
|
5
|
+
It finds why each failing test failed (a real bug, a flaky test, the environment, test data, or a changed selector), says whether a test could ever fail at all, breaks your code to find out whether the tests would notice, reads what a migration will do to a live database, and gives one verdict on whether a change is done. Every answer carries its evidence and names what decided it: a deterministic rule, the AI model, or a person's correction.
|
|
6
|
+
|
|
7
|
+
What makes it worth installing is what it refuses to say. It answers *"cannot say"* rather than guessing, it will not narrow a test run it cannot prove safe to narrow, and every report ends with what it did **not** check. Each number below was measured on strangers' real code, never on fixtures.
|
|
4
8
|
|
|
5
9
|
It works for every stack: Node.js (node:test, Jest, Vitest, Mocha), Python (pytest, unittest), Java and Kotlin (JUnit, TestNG), Go, .NET, Ruby, PHP and Rust, each checked against its real runner's report ([the full list](QUICKSTART.md#not-using-playwright-every-stack-works)), plus Playwright in depth with traces, selector drift and verified fixes. It also reads the failure wording of the browser and component-test tools on top of those runners (Selenium, WebdriverIO, Cypress, Testing Library for React, Vue and Angular, Vue Test Utils, jest-dom and Jasmine), checked against more than 500 real messages from public projects.
|
|
6
10
|
|
|
@@ -112,7 +116,25 @@ jobs:
|
|
|
112
116
|
|
|
113
117
|
The report reads the history cache of the branch it runs on, usually the default branch.
|
|
114
118
|
|
|
115
|
-
The repository needs "Allow GitHub Actions to create and approve pull requests" enabled for fix PRs. Pull requests opened by the workflow token don't trigger CI themselves; the agent verified each change by re-running the test before opening the PR.
|
|
119
|
+
The repository needs "Allow GitHub Actions to create and approve pull requests" enabled for fix PRs. Pull requests opened by the workflow token don't trigger CI themselves; the agent verified each change by re-running the test before opening the PR. It was first proved end to end on a private practice repository, which is why there is no link to click here. If this repository is private, other repositories can only use the action when Settings → Actions → Access allows it.
|
|
120
|
+
|
|
121
|
+
### As a Claude Code plugin (no account, no marketplace approval)
|
|
122
|
+
|
|
123
|
+
```bash
|
|
124
|
+
claude plugin marketplace add hmamut39/tewip-plugin
|
|
125
|
+
claude plugin install tewip@tewip
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
That is the whole publication process. A marketplace is a `.claude-plugin/marketplace.json` file in a git
|
|
129
|
+
repository, so Tewip publishes one at
|
|
130
|
+
[hmamut39/tewip-plugin](https://github.com/hmamut39/tewip-plugin) - nobody has to approve it, and there
|
|
131
|
+
is nothing to pay for. That repository holds the plugin only: a manifest, three skills and a pointer to
|
|
132
|
+
the npm package, which is why this one can stay private. The plugin brings Tewip's MCP server and three skills that tell Claude Code *when* to use it:
|
|
133
|
+
before claiming a change is done, when a test fails, and when a change touches a migration. It costs
|
|
134
|
+
about 410 tokens per session until a skill is actually needed.
|
|
135
|
+
|
|
136
|
+
The MCP server runs from the published package (`npx -y tewip-ai@latest mcp`), so the plugin stays small
|
|
137
|
+
and you always get the current version.
|
|
116
138
|
|
|
117
139
|
### The editor extension (VS Code, Cursor, Windsurf, VSCodium)
|
|
118
140
|
|
|
@@ -801,7 +823,7 @@ With `--act` it carries the decision out, and this is where the refusals matter
|
|
|
801
823
|
|
|
802
824
|
| Project | How | Result |
|
|
803
825
|
|---|---|---|
|
|
804
|
-
|
|
|
826
|
+
| A private practice repository (not linked: it is private) | Live GitHub Action on a release PR with 11 planted failures | Diagnosis comment, one environment issue, and a fix PR whose 5 locator changes pass when checked out. Traps left alone. Synthetic. |
|
|
805
827
|
| WordPress Gutenberg (public) | Offline, from 5 windows of real CI blob reports, two of them harvested after the pipeline was frozen | 84–98% accuracy on 1,441 labelled real failures; the only project with enough drift cases to measure that category (see [eval/README.md](eval/README.md) for caveats) |
|
|
806
828
|
| JupyterLab (public) | Offline, from its Galata UI suite | 90.2% on 41 labelled failures; the deterministic rules alone were right on 34 of 34 they decided |
|
|
807
829
|
| LXD UI (public) | Offline, from its pull-request suite | 91.3% on 69 labelled failures; rules right on 63 of 63. All 6 misses are cross-branch flakes read as real bugs |
|
package/dist/cli/tewip.js
CHANGED
|
@@ -12,6 +12,7 @@ import { homedir } from 'node:os';
|
|
|
12
12
|
import { join, resolve } from 'node:path';
|
|
13
13
|
import { flag } from './args.js';
|
|
14
14
|
import { HELP } from './help.js';
|
|
15
|
+
import { proveVerb } from './verbs-prove.js';
|
|
15
16
|
import { workVerb } from './verbs-work.js';
|
|
16
17
|
// cli/ in the source tree, dist/cli/ in the published package: package.json is one or two levels up.
|
|
17
18
|
const ROOT = [join(import.meta.dirname, '..'), join(import.meta.dirname, '..', '..')].find((d) => existsSync(join(d, 'package.json'))) ?? join(import.meta.dirname, '..');
|
|
@@ -21,6 +22,9 @@ async function main() {
|
|
|
21
22
|
// The verbs that reach into a tracker, a scan or a build live next door; the rest are here.
|
|
22
23
|
if (await workVerb(verb, args))
|
|
23
24
|
return;
|
|
25
|
+
// The verbs that prove a change live next door too.
|
|
26
|
+
if (await proveVerb(verb, args))
|
|
27
|
+
return;
|
|
24
28
|
switch (verb) {
|
|
25
29
|
case 'explain': {
|
|
26
30
|
// The explain script reads its arguments from process.argv, as when it is run directly.
|
|
@@ -185,75 +189,6 @@ async function main() {
|
|
|
185
189
|
runPursue({ root: process.cwd(), goal, act: args.includes('--act'), name: flag(args, 'name'), out: flag(args, 'out') });
|
|
186
190
|
return;
|
|
187
191
|
}
|
|
188
|
-
case 'done': {
|
|
189
|
-
const { runDone } = await import('../engine/verdict/verdict-command.js');
|
|
190
|
-
runDone({
|
|
191
|
-
root: process.cwd(),
|
|
192
|
-
diff: flag(args, 'diff') ?? 'main...HEAD',
|
|
193
|
-
prove: args.includes('--prove'),
|
|
194
|
-
proveBudgetSeconds: Number(flag(args, 'budget') ?? '300'),
|
|
195
|
-
out: flag(args, 'out'),
|
|
196
|
-
json: flag(args, 'json'),
|
|
197
|
-
name: flag(args, 'name'),
|
|
198
|
-
strict: args.includes('--strict'),
|
|
199
|
-
});
|
|
200
|
-
return;
|
|
201
|
-
}
|
|
202
|
-
case 'standards': {
|
|
203
|
-
const { runStandards } = await import('../engine/standards/standards-command.js');
|
|
204
|
-
runStandards({
|
|
205
|
-
root: process.cwd(),
|
|
206
|
-
diff: flag(args, 'diff'),
|
|
207
|
-
files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out'),
|
|
208
|
-
out: flag(args, 'out'),
|
|
209
|
-
strict: args.includes('--strict'),
|
|
210
|
-
});
|
|
211
|
-
return;
|
|
212
|
-
}
|
|
213
|
-
case 'schema': {
|
|
214
|
-
const { runSchema } = await import('../engine/schema/schema-command.js');
|
|
215
|
-
runSchema({
|
|
216
|
-
root: process.cwd(),
|
|
217
|
-
diff: flag(args, 'diff') ?? 'main...HEAD',
|
|
218
|
-
files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out'),
|
|
219
|
-
out: flag(args, 'out'),
|
|
220
|
-
strict: args.includes('--strict'),
|
|
221
|
-
});
|
|
222
|
-
return;
|
|
223
|
-
}
|
|
224
|
-
case 'pick': {
|
|
225
|
-
const { runPick } = await import('../engine/pick/pick-command.js');
|
|
226
|
-
runPick({
|
|
227
|
-
root: process.cwd(),
|
|
228
|
-
diff: flag(args, 'diff') ?? 'main...HEAD',
|
|
229
|
-
files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out' && args[i - 1] !== '--recent' && args[i - 1] !== '--name'),
|
|
230
|
-
out: flag(args, 'out'),
|
|
231
|
-
list: args.includes('--list'),
|
|
232
|
-
recentRuns: Number(flag(args, 'recent') ?? '5'),
|
|
233
|
-
name: flag(args, 'name'),
|
|
234
|
-
});
|
|
235
|
-
return;
|
|
236
|
-
}
|
|
237
|
-
case 'prove': {
|
|
238
|
-
const paths = args.filter((a, i) => !a.startsWith('--') && !['--out', '--diff', '--limit', '--budget'].includes(args[i - 1] ?? ''));
|
|
239
|
-
const { runProve } = await import('../engine/prove/prove-command.js');
|
|
240
|
-
runProve({
|
|
241
|
-
root: process.cwd(),
|
|
242
|
-
files: paths,
|
|
243
|
-
diff: flag(args, 'diff'),
|
|
244
|
-
limit: Number(flag(args, 'limit') ?? '6'),
|
|
245
|
-
budgetSeconds: Number(flag(args, 'budget') ?? '600'),
|
|
246
|
-
out: flag(args, 'out'),
|
|
247
|
-
strict: args.includes('--strict'),
|
|
248
|
-
});
|
|
249
|
-
return;
|
|
250
|
-
}
|
|
251
|
-
case 'vouch': {
|
|
252
|
-
const paths = args.filter((a, i) => !a.startsWith('--') && !['--out', '--json'].includes(args[i - 1] ?? ''));
|
|
253
|
-
const { runVouch } = await import('../engine/vouch/run-vouch.js');
|
|
254
|
-
runVouch({ root: process.cwd(), paths: paths.length > 0 ? paths : ['.'], out: flag(args, 'out'), json: flag(args, 'json'), strict: args.includes('--strict') });
|
|
255
|
-
return;
|
|
256
|
-
}
|
|
257
192
|
case 'trace': {
|
|
258
193
|
const { runTrace } = await import('../engine/trace/trace-command.js');
|
|
259
194
|
runTrace({ root: process.cwd(), only: flag(args, 'only'), name: flag(args, 'name'), out: flag(args, 'out') });
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
// The verbs that prove a change rather than describe it: the verdict, the project's own standards,
|
|
2
|
+
// what a migration does to a live database, which tests could notice a change, whether the tests would
|
|
3
|
+
// catch a bug, and whether a test could ever fail. Split out of cli/tewip.ts the same way the work
|
|
4
|
+
// verbs were, to keep every file inside this project's own 300-line rule.
|
|
5
|
+
import { flag } from './args.js';
|
|
6
|
+
/** Runs `verb` if it is one of these, and says whether it did. */
|
|
7
|
+
export async function proveVerb(verb, args) {
|
|
8
|
+
switch (verb) {
|
|
9
|
+
case 'done': {
|
|
10
|
+
const { runDone } = await import('../engine/verdict/verdict-command.js');
|
|
11
|
+
runDone({
|
|
12
|
+
root: process.cwd(),
|
|
13
|
+
diff: flag(args, 'diff') ?? 'main...HEAD',
|
|
14
|
+
prove: args.includes('--prove'),
|
|
15
|
+
proveBudgetSeconds: Number(flag(args, 'budget') ?? '300'),
|
|
16
|
+
out: flag(args, 'out'),
|
|
17
|
+
json: flag(args, 'json'),
|
|
18
|
+
name: flag(args, 'name'),
|
|
19
|
+
strict: args.includes('--strict'),
|
|
20
|
+
});
|
|
21
|
+
return true;
|
|
22
|
+
}
|
|
23
|
+
case 'standards': {
|
|
24
|
+
const { runStandards } = await import('../engine/standards/standards-command.js');
|
|
25
|
+
runStandards({
|
|
26
|
+
root: process.cwd(),
|
|
27
|
+
diff: flag(args, 'diff'),
|
|
28
|
+
files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out'),
|
|
29
|
+
out: flag(args, 'out'),
|
|
30
|
+
strict: args.includes('--strict'),
|
|
31
|
+
});
|
|
32
|
+
return true;
|
|
33
|
+
}
|
|
34
|
+
case 'schema': {
|
|
35
|
+
const { runSchema } = await import('../engine/schema/schema-command.js');
|
|
36
|
+
runSchema({
|
|
37
|
+
root: process.cwd(),
|
|
38
|
+
diff: flag(args, 'diff') ?? 'main...HEAD',
|
|
39
|
+
files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out'),
|
|
40
|
+
out: flag(args, 'out'),
|
|
41
|
+
strict: args.includes('--strict'),
|
|
42
|
+
});
|
|
43
|
+
return true;
|
|
44
|
+
}
|
|
45
|
+
case 'pick': {
|
|
46
|
+
const { runPick } = await import('../engine/pick/pick-command.js');
|
|
47
|
+
runPick({
|
|
48
|
+
root: process.cwd(),
|
|
49
|
+
diff: flag(args, 'diff') ?? 'main...HEAD',
|
|
50
|
+
files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out' && args[i - 1] !== '--recent' && args[i - 1] !== '--name'),
|
|
51
|
+
out: flag(args, 'out'),
|
|
52
|
+
list: args.includes('--list'),
|
|
53
|
+
recentRuns: Number(flag(args, 'recent') ?? '5'),
|
|
54
|
+
name: flag(args, 'name'),
|
|
55
|
+
});
|
|
56
|
+
return true;
|
|
57
|
+
}
|
|
58
|
+
case 'prove': {
|
|
59
|
+
const paths = args.filter((a, i) => !a.startsWith('--') && !['--out', '--diff', '--limit', '--budget'].includes(args[i - 1] ?? ''));
|
|
60
|
+
const { runProve } = await import('../engine/prove/prove-command.js');
|
|
61
|
+
runProve({
|
|
62
|
+
root: process.cwd(),
|
|
63
|
+
files: paths,
|
|
64
|
+
diff: flag(args, 'diff'),
|
|
65
|
+
limit: Number(flag(args, 'limit') ?? '6'),
|
|
66
|
+
budgetSeconds: Number(flag(args, 'budget') ?? '600'),
|
|
67
|
+
out: flag(args, 'out'),
|
|
68
|
+
strict: args.includes('--strict'),
|
|
69
|
+
});
|
|
70
|
+
return true;
|
|
71
|
+
}
|
|
72
|
+
case 'vouch': {
|
|
73
|
+
const paths = args.filter((a, i) => !a.startsWith('--') && !['--out', '--json'].includes(args[i - 1] ?? ''));
|
|
74
|
+
const { runVouch } = await import('../engine/vouch/run-vouch.js');
|
|
75
|
+
runVouch({ root: process.cwd(), paths: paths.length > 0 ? paths : ['.'], out: flag(args, 'out'), json: flag(args, 'json'), strict: args.includes('--strict') });
|
|
76
|
+
return true;
|
|
77
|
+
}
|
|
78
|
+
default:
|
|
79
|
+
return false;
|
|
80
|
+
}
|
|
81
|
+
return true;
|
|
82
|
+
}
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
/** A passing attempt: the same object every time, since it carries nothing.
|
|
2
|
+
*/
|
|
3
|
+
export const PASSED = { failed: false, message: '', stack: '' };
|
|
4
|
+
/** One failed attempt, read out of whatever the runner wrote. */
|
|
5
|
+
export function failure(rawMessage, rawDetail) {
|
|
6
|
+
// Terminal colour codes: XML cannot hold the escape character, so runners often write what is left
|
|
7
|
+
// of them ("[2mexpect([22m"); both forms are noise to the rules.
|
|
8
|
+
const plain = (s) => s.replace(/\u001b?\[(?:\d{1,3};)*\d{1,3}m/g, '');
|
|
9
|
+
const message = plain(rawMessage);
|
|
10
|
+
const detail = plain(rawDetail);
|
|
11
|
+
const head = message.trim() || detail.split('\n').find((l) => l.trim() !== '') || 'failed';
|
|
12
|
+
const stack = detail.trim();
|
|
13
|
+
// The full detail goes into the message too: outside Playwright the useful part (the assertion,
|
|
14
|
+
// the exception) is usually in the detail, and the rules read the message.
|
|
15
|
+
return { failed: true, message: stack && !stack.includes(head) ? `${head}\n${stack}` : stack || head, stack };
|
|
16
|
+
}
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
// The .NET report formats: Visual Studio's own TRX, and NUnit 3's XML.
|
|
2
|
+
//
|
|
3
|
+
// Both say the same things as a JUnit report in different words, so they are read into the same `Case`
|
|
4
|
+
// shape and the rest of the pipeline never learns the difference. Split out of junit.ts to keep that
|
|
5
|
+
// file inside this project's 300-line rule; nothing else changed.
|
|
6
|
+
import { PASSED, failure } from './case.js';
|
|
7
|
+
import { descendants } from './xml.js';
|
|
8
|
+
/** "00:00:01.2340000" -> 1234 ms. */
|
|
9
|
+
const trxDuration = (d) => {
|
|
10
|
+
const m = d?.match(/^(\d+):(\d+):(\d+(?:\.\d+)?)$/);
|
|
11
|
+
return m ? Math.round(((Number(m[1]) * 60 + Number(m[2])) * 60 + Number(m[3])) * 1000) : 0;
|
|
12
|
+
};
|
|
13
|
+
/** .NET's native TRX report (dotnet test --logger trx). */
|
|
14
|
+
export function trxCases(root) {
|
|
15
|
+
const methods = new Map();
|
|
16
|
+
for (const unit of descendants(root, 'UnitTest')) {
|
|
17
|
+
const method = unit.children.find((c) => c.name === 'TestMethod');
|
|
18
|
+
if (unit.attrs.id && method)
|
|
19
|
+
methods.set(unit.attrs.id, { className: method.attrs.className ?? '', name: method.attrs.name ?? '' });
|
|
20
|
+
}
|
|
21
|
+
return descendants(root, 'UnitTestResult')
|
|
22
|
+
.filter((r) => r.attrs.outcome !== 'NotExecuted')
|
|
23
|
+
.map((r) => {
|
|
24
|
+
const method = methods.get(r.attrs.testId ?? '');
|
|
25
|
+
const testId = method?.className ? `${method.className} > ${method.name}` : (r.attrs.testName ?? '').trim();
|
|
26
|
+
const info = descendants(r, 'ErrorInfo')[0];
|
|
27
|
+
const text = (name) => info?.children.find((c) => c.name === name)?.text ?? '';
|
|
28
|
+
const failed = r.attrs.outcome === 'Failed' || r.attrs.outcome === 'Error' || r.attrs.outcome === 'Timeout';
|
|
29
|
+
return {
|
|
30
|
+
testId,
|
|
31
|
+
file: method?.className ?? testId,
|
|
32
|
+
duration: trxDuration(r.attrs.duration),
|
|
33
|
+
attempts: [failed ? failure(text('Message'), text('StackTrace')) : PASSED],
|
|
34
|
+
};
|
|
35
|
+
});
|
|
36
|
+
}
|
|
37
|
+
/**
|
|
38
|
+
* NUnit 3's own result file (`nunit3-console --result=x.xml`), whose root is <test-run>. NUnit does
|
|
39
|
+
* not write JUnit XML at all - its console offers only the nunit3 and nunit2 formats - so a .NET
|
|
40
|
+
* team on NUnit has nothing else to hand Tewip.
|
|
41
|
+
*/
|
|
42
|
+
export function nunitCases(root) {
|
|
43
|
+
return descendants(root, 'test-case')
|
|
44
|
+
.filter((t) => t.attrs.result !== 'Skipped' && t.attrs.result !== 'Inconclusive')
|
|
45
|
+
.map((t) => {
|
|
46
|
+
const failureEl = t.children.find((c) => c.name === 'failure');
|
|
47
|
+
const child = (parent, name) => parent?.children.find((c) => c.name === name)?.text ?? '';
|
|
48
|
+
const testId = (t.attrs.fullname ?? '').trim() || [t.attrs.classname, t.attrs.methodname ?? t.attrs.name].filter(Boolean).join(' > ');
|
|
49
|
+
const seconds = Number(t.attrs.duration);
|
|
50
|
+
return {
|
|
51
|
+
testId,
|
|
52
|
+
file: (t.attrs.classname ?? '').trim(),
|
|
53
|
+
duration: Number.isFinite(seconds) ? Math.round(seconds * 1000) : 0,
|
|
54
|
+
attempts: [t.attrs.result === 'Failed' ? failure(child(failureEl, 'message'), child(failureEl, 'stack-trace')) : PASSED],
|
|
55
|
+
};
|
|
56
|
+
});
|
|
57
|
+
}
|
|
58
|
+
// .jtl is JMeter's, and a k6 summary is JSON; both are read below.
|
|
@@ -14,23 +14,12 @@ import { join } from 'node:path';
|
|
|
14
14
|
import { SCHEMA_VERSION } from '../../reporters/types.js';
|
|
15
15
|
import { descendants, parseXml } from './xml.js';
|
|
16
16
|
import { jmeterCases, k6Cases, looksLikeK6, robotCases, robotTime } from './other-formats.js';
|
|
17
|
-
|
|
17
|
+
import { nunitCases, trxCases } from './dotnet.js';
|
|
18
|
+
import { PASSED, failure } from './case.js';
|
|
18
19
|
const FAILURE = new Set(['failure', 'error']);
|
|
19
20
|
/** Maven Surefire's records of earlier attempts: flaky* when a later one passed, rerun* when not. */
|
|
20
21
|
const RETRY_FAILURE = new Set(['flakyFailure', 'flakyError', 'rerunFailure', 'rerunError']);
|
|
21
22
|
/** A failure's message and detail; runners disagree on which of the two holds what. */
|
|
22
|
-
function failure(rawMessage, rawDetail) {
|
|
23
|
-
// Terminal colour codes: XML cannot hold the escape character, so runners often write what is left
|
|
24
|
-
// of them ("[2mexpect([22m"); both forms are noise to the rules.
|
|
25
|
-
const plain = (s) => s.replace(/\u001b?\[(?:\d{1,3};)*\d{1,3}m/g, '');
|
|
26
|
-
const message = plain(rawMessage);
|
|
27
|
-
const detail = plain(rawDetail);
|
|
28
|
-
const head = message.trim() || detail.split('\n').find((l) => l.trim() !== '') || 'failed';
|
|
29
|
-
const stack = detail.trim();
|
|
30
|
-
// The full detail goes into the message too: outside Playwright the useful part (the assertion,
|
|
31
|
-
// the exception) is usually in the detail, and the rules read the message.
|
|
32
|
-
return { failed: true, message: stack && !stack.includes(head) ? `${head}\n${stack}` : stack || head, stack };
|
|
33
|
-
}
|
|
34
23
|
const fromElement = (e) => failure(e.attrs.message ?? e.attrs.type ?? '', e.children.find((c) => c.name === 'stackTrace')?.text ?? e.text);
|
|
35
24
|
/** The attempts one <testcase> records: earlier failed retries first, then the final outcome. */
|
|
36
25
|
function attemptsOf(testcase) {
|
|
@@ -130,57 +119,6 @@ function junitCases(root, workspace) {
|
|
|
130
119
|
}
|
|
131
120
|
return out;
|
|
132
121
|
}
|
|
133
|
-
/** "00:00:01.2340000" -> 1234 ms. */
|
|
134
|
-
const trxDuration = (d) => {
|
|
135
|
-
const m = d?.match(/^(\d+):(\d+):(\d+(?:\.\d+)?)$/);
|
|
136
|
-
return m ? Math.round(((Number(m[1]) * 60 + Number(m[2])) * 60 + Number(m[3])) * 1000) : 0;
|
|
137
|
-
};
|
|
138
|
-
/** .NET's native TRX report (dotnet test --logger trx). */
|
|
139
|
-
function trxCases(root) {
|
|
140
|
-
const methods = new Map();
|
|
141
|
-
for (const unit of descendants(root, 'UnitTest')) {
|
|
142
|
-
const method = unit.children.find((c) => c.name === 'TestMethod');
|
|
143
|
-
if (unit.attrs.id && method)
|
|
144
|
-
methods.set(unit.attrs.id, { className: method.attrs.className ?? '', name: method.attrs.name ?? '' });
|
|
145
|
-
}
|
|
146
|
-
return descendants(root, 'UnitTestResult')
|
|
147
|
-
.filter((r) => r.attrs.outcome !== 'NotExecuted')
|
|
148
|
-
.map((r) => {
|
|
149
|
-
const method = methods.get(r.attrs.testId ?? '');
|
|
150
|
-
const testId = method?.className ? `${method.className} > ${method.name}` : (r.attrs.testName ?? '').trim();
|
|
151
|
-
const info = descendants(r, 'ErrorInfo')[0];
|
|
152
|
-
const text = (name) => info?.children.find((c) => c.name === name)?.text ?? '';
|
|
153
|
-
const failed = r.attrs.outcome === 'Failed' || r.attrs.outcome === 'Error' || r.attrs.outcome === 'Timeout';
|
|
154
|
-
return {
|
|
155
|
-
testId,
|
|
156
|
-
file: method?.className ?? testId,
|
|
157
|
-
duration: trxDuration(r.attrs.duration),
|
|
158
|
-
attempts: [failed ? failure(text('Message'), text('StackTrace')) : PASSED],
|
|
159
|
-
};
|
|
160
|
-
});
|
|
161
|
-
}
|
|
162
|
-
/**
|
|
163
|
-
* NUnit 3's own result file (`nunit3-console --result=x.xml`), whose root is <test-run>. NUnit does
|
|
164
|
-
* not write JUnit XML at all - its console offers only the nunit3 and nunit2 formats - so a .NET
|
|
165
|
-
* team on NUnit has nothing else to hand Tewip.
|
|
166
|
-
*/
|
|
167
|
-
function nunitCases(root) {
|
|
168
|
-
return descendants(root, 'test-case')
|
|
169
|
-
.filter((t) => t.attrs.result !== 'Skipped' && t.attrs.result !== 'Inconclusive')
|
|
170
|
-
.map((t) => {
|
|
171
|
-
const failureEl = t.children.find((c) => c.name === 'failure');
|
|
172
|
-
const child = (parent, name) => parent?.children.find((c) => c.name === name)?.text ?? '';
|
|
173
|
-
const testId = (t.attrs.fullname ?? '').trim() || [t.attrs.classname, t.attrs.methodname ?? t.attrs.name].filter(Boolean).join(' > ');
|
|
174
|
-
const seconds = Number(t.attrs.duration);
|
|
175
|
-
return {
|
|
176
|
-
testId,
|
|
177
|
-
file: (t.attrs.classname ?? '').trim(),
|
|
178
|
-
duration: Number.isFinite(seconds) ? Math.round(seconds * 1000) : 0,
|
|
179
|
-
attempts: [t.attrs.result === 'Failed' ? failure(child(failureEl, 'message'), child(failureEl, 'stack-trace')) : PASSED],
|
|
180
|
-
};
|
|
181
|
-
});
|
|
182
|
-
}
|
|
183
|
-
// .jtl is JMeter's, and a k6 summary is JSON; both are read below.
|
|
184
122
|
const REPORT_FILE = /\.(xml|trx|jtl|json)$/i;
|
|
185
123
|
/** Report files under `path` (a file, or a directory searched recursively). */
|
|
186
124
|
export function junitFiles(path) {
|
package/dist/mcp/server.js
CHANGED
|
@@ -5,14 +5,29 @@
|
|
|
5
5
|
// Read-only by design, except test_plan, which saves the plan it returns in the project's .tewip
|
|
6
6
|
// folder: no tool changes code, CI, GitHub or collected data.
|
|
7
7
|
// Run: node mcp/server.ts (stdio). Setup for each client is in README.md.
|
|
8
|
+
import { registerProofTools } from './tools-prove.js';
|
|
8
9
|
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
|
|
9
10
|
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
|
10
11
|
import { existsSync } from 'node:fs';
|
|
11
|
-
import {
|
|
12
|
+
import { readFileSync } from 'node:fs';
|
|
13
|
+
import { isAbsolute, join } from 'node:path';
|
|
12
14
|
import { z } from 'zod';
|
|
13
15
|
import { githubCiReport } from './github-report.js';
|
|
14
16
|
import { explainFailure, listDatasets, listFailedRuns, runView, windowView } from './tools.js';
|
|
15
|
-
|
|
17
|
+
/**
|
|
18
|
+
* The version a client is told, read from the package rather than written here. It had drifted to 0.5.0
|
|
19
|
+
* while the package was at 0.32.0 - a small lie, but this server's whole job is to be the thing that
|
|
20
|
+
* does not say unverified numbers.
|
|
21
|
+
*/
|
|
22
|
+
const version = (() => {
|
|
23
|
+
try {
|
|
24
|
+
return JSON.parse(readFileSync(join(import.meta.dirname, '..', 'package.json'), 'utf8')).version ?? '0.0.0';
|
|
25
|
+
}
|
|
26
|
+
catch {
|
|
27
|
+
return '0.0.0';
|
|
28
|
+
}
|
|
29
|
+
})();
|
|
30
|
+
const server = new McpServer({ name: 'tewip', version });
|
|
16
31
|
const readOnly = { readOnlyHint: true, destructiveHint: false, idempotentHint: true };
|
|
17
32
|
const dataset = z.string().optional().describe('Dataset directory relative to the Tewip repo, e.g. "data/real/gutenberg-holdout". Default: data/ (what the reporter collected). See list_datasets.');
|
|
18
33
|
const run = z.string().optional().describe('Run id or a unique part of it. Default: the latest run with failures.');
|
|
@@ -123,115 +138,5 @@ server.registerTool('next_action', {
|
|
|
123
138
|
const observed = observe(root, goal);
|
|
124
139
|
return renderPursuit(goal, observed, decide(goal, observed), false);
|
|
125
140
|
}));
|
|
126
|
-
server
|
|
127
|
-
title: 'Could these tests ever fail?',
|
|
128
|
-
description: "Reads test files and says, per test, whether a change to the code could break them: which compare no value, which leave an expectation unfinished, which compare something with itself, which catch and discard their own failure, and which never run. Where the checking happens in a helper it cannot follow it says so instead of accusing. Costs nothing and uses no model. Use it on tests that were just written - including tests you wrote yourself - before claiming a change is covered.",
|
|
129
|
-
inputSchema: {
|
|
130
|
-
project,
|
|
131
|
-
paths: z.array(z.string()).optional().describe('Paths inside the project to read, relative to it. Defaults to the whole project.'),
|
|
132
|
-
},
|
|
133
|
-
annotations: readOnly,
|
|
134
|
-
}, ({ project: root, paths }) => answer(async () => {
|
|
135
|
-
if (!isAbsolute(root) || !existsSync(root))
|
|
136
|
-
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
137
|
-
const { vouchReport } = await import('../engine/vouch/run-vouch.js');
|
|
138
|
-
return vouchReport(root, paths && paths.length > 0 ? paths : ['.']).report;
|
|
139
|
-
}));
|
|
140
|
-
server.registerTool('prove_tests_catch_bugs', {
|
|
141
|
-
title: 'Would the tests catch a bug here?',
|
|
142
|
-
description: "Breaks one source file in small, deliberate ways - a boundary moved by one, a condition inverted, an and that became an or, a returned flag flipped - runs the tests that reach that file, and reports every change none of them noticed. This is execution, not opinion: the line was covered, it just was not checked. Slow by nature (each change means a real test run, so expect minutes) and it needs a unit test framework. The file is always put back exactly as it was, and if the tests turn out not to load the file at all it says so instead of reporting false holes. Use it after writing or changing code, to find out whether the tests you are relying on would have stopped a mistake.",
|
|
143
|
-
inputSchema: {
|
|
144
|
-
project,
|
|
145
|
-
file: z.string().describe('One source file, relative to the project, e.g. src/pricing.js'),
|
|
146
|
-
limit: z.number().optional().describe('How many changes to try (default 6). More is a better answer and a longer wait.'),
|
|
147
|
-
budgetSeconds: z.number().optional().describe('Total seconds to spend before stopping and saying how many were left (default 600)'),
|
|
148
|
-
},
|
|
149
|
-
annotations: { readOnlyHint: false, destructiveHint: false, idempotentHint: true, openWorldHint: false },
|
|
150
|
-
}, ({ project: root, file, limit, budgetSeconds }) => answer(async () => {
|
|
151
|
-
if (!isAbsolute(root) || !existsSync(root))
|
|
152
|
-
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
153
|
-
// runProve prints its report; over MCP the report is the return value and printing would corrupt
|
|
154
|
-
// the protocol, so the pieces are called directly.
|
|
155
|
-
const { proveOne } = await import('../engine/prove/prove-command.js');
|
|
156
|
-
return proveOne(root, file, { limit, budgetSeconds });
|
|
157
|
-
}));
|
|
158
|
-
server.registerTool('is_the_work_done', {
|
|
159
|
-
title: 'Is this work actually done?',
|
|
160
|
-
description: "One verdict on a change, from the layers that each have their own measured accuracy: is anything failing, is the changed code tested by something that really reaches it, can the tests in the change fail at all, would they catch a bug (only when prove is asked for, because that costs minutes), and is what was asked for covered. Answers ready, not ready, or cannot say - and cannot say is never rounded up to ready. Use it before claiming a piece of work is finished, including your own: a green test run is not the same as work that is done.",
|
|
161
|
-
inputSchema: {
|
|
162
|
-
project,
|
|
163
|
-
diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
|
|
164
|
-
prove: z.boolean().optional().describe('Also break the changed code and run the tests against it. Costs minutes.'),
|
|
165
|
-
},
|
|
166
|
-
annotations: readOnly,
|
|
167
|
-
}, ({ project: root, diff, prove }) => answer(async () => {
|
|
168
|
-
if (!isAbsolute(root) || !existsSync(root))
|
|
169
|
-
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
170
|
-
// gather/decide/render are pure; runDone prints, which would corrupt the protocol.
|
|
171
|
-
const { gather } = await import('../engine/verdict/verdict-command.js');
|
|
172
|
-
const { decide, renderDecision } = await import('../engine/verdict/verdict.js');
|
|
173
|
-
const range = diff ?? 'main...HEAD';
|
|
174
|
-
const gathered = gather({ root, diff: range, prove });
|
|
175
|
-
if (!gathered)
|
|
176
|
-
return `Nothing changed in ${range}, or that range could not be read here.`;
|
|
177
|
-
return renderDecision(range, decide(gathered.evidence));
|
|
178
|
-
}));
|
|
179
|
-
server.registerTool('which_tests_to_run', {
|
|
180
|
-
title: 'Which tests could notice this change?',
|
|
181
|
-
description: "Narrows a test run to the tests that could notice a change, from facts in the source rather than a prediction: a test runs when it imports a changed file, reaches one through another module, is itself edited by the change, reads the repository at run time, or has failed recently. It refuses to narrow at all - and says why - when the change touches dependencies, build or test configuration, shared setup or fixtures, when the suite drives a browser, or when a changed file cannot be traced to any test. Use it to run the right tests quickly; what it leaves out, it leaves out for a stated reason.",
|
|
182
|
-
inputSchema: {
|
|
183
|
-
project,
|
|
184
|
-
diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
|
|
185
|
-
files: z.array(z.string()).optional().describe('The changed files, when they do not come from a git range'),
|
|
186
|
-
},
|
|
187
|
-
annotations: readOnly,
|
|
188
|
-
}, ({ project: root, diff, files }) => answer(async () => {
|
|
189
|
-
if (!isAbsolute(root) || !existsSync(root))
|
|
190
|
-
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
191
|
-
// choose/renderPick are pure; runPick prints, which would corrupt the protocol.
|
|
192
|
-
const { choose } = await import('../engine/pick/pick-command.js');
|
|
193
|
-
const { renderPick } = await import('../engine/pick/pick.js');
|
|
194
|
-
const range = diff ?? 'main...HEAD';
|
|
195
|
-
const chosen = choose({ root, diff: range, files });
|
|
196
|
-
if (!chosen)
|
|
197
|
-
return `Nothing changed in ${range}, or that range could not be read here.`;
|
|
198
|
-
return renderPick(chosen.picked, chosen.total, chosen.command, chosen.noHistory);
|
|
199
|
-
}));
|
|
200
|
-
server.registerTool('what_will_this_migration_do', {
|
|
201
|
-
title: 'What will this migration do to a live database?',
|
|
202
|
-
description: "Reads the migrations in a change and names what will happen when they run against a live database: a column dropped while the version still running keeps selecting it, a table dropped, an index built under a lock that blocks writes, a NOT NULL added to a populated table, a type change that rewrites it, a foreign key validated against every row, a data migration holding a transaction open, a step that cannot be undone. It also searches the repository for code that still uses the column being taken away. Reads SQL, Django, Rails and Alembic, judges only the forward path, and says plainly when it cannot read a migration. It does not know your table sizes, so it says what a statement does rather than how long it will take.",
|
|
203
|
-
inputSchema: {
|
|
204
|
-
project,
|
|
205
|
-
diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
|
|
206
|
-
files: z.array(z.string()).optional().describe('Migration files to read, when they do not come from a git range'),
|
|
207
|
-
},
|
|
208
|
-
annotations: readOnly,
|
|
209
|
-
}, ({ project: root, diff, files }) => answer(async () => {
|
|
210
|
-
if (!isAbsolute(root) || !existsSync(root))
|
|
211
|
-
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
212
|
-
const { readMigration, renderSchema, schemaFiles, stillUsed } = await import('../engine/schema/schema-command.js');
|
|
213
|
-
const cmd = { root, diff: diff ?? 'main...HEAD', files };
|
|
214
|
-
const migrations = schemaFiles(cmd).map((file) => readMigration(root, file));
|
|
215
|
-
if (migrations.length === 0)
|
|
216
|
-
return 'No migration or schema file in that change.';
|
|
217
|
-
return renderSchema(migrations, stillUsed(root, migrations.flatMap((m) => m.findings)));
|
|
218
|
-
}));
|
|
219
|
-
server.registerTool('project_standards', {
|
|
220
|
-
title: "What are this project's own rules?",
|
|
221
|
-
description: "The house rules this team has written down in .tewip/standards/, and whether a change keeps them. Returns two things kept carefully apart: rules Tewip checks and enforces (a forbid or require pattern over a set of files), and guidance it can only carry - prose for the parts no pattern captures, which is passed on when something is written but never reported as verified. Use it before writing or changing code in somebody's project, so what you produce matches their conventions rather than yours.",
|
|
222
|
-
inputSchema: {
|
|
223
|
-
project,
|
|
224
|
-
files: z.array(z.string()).optional().describe('Files to check against the rules. Omit to list what the project has written.'),
|
|
225
|
-
diff: z.string().optional().describe('A git range whose changed files should be checked instead'),
|
|
226
|
-
},
|
|
227
|
-
annotations: readOnly,
|
|
228
|
-
}, ({ project: root, files, diff }) => answer(async () => {
|
|
229
|
-
if (!isAbsolute(root) || !existsSync(root))
|
|
230
|
-
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
231
|
-
const { checkChanged, loadStandards, renderStandards } = await import('../engine/standards/standards-command.js');
|
|
232
|
-
const { changeSummary } = await import('../engine/quality/grounding.js');
|
|
233
|
-
const standards = loadStandards(root);
|
|
234
|
-
const changed = files && files.length > 0 ? files : diff ? (changeSummary(root, diff)?.files ?? []) : [];
|
|
235
|
-
return renderStandards(standards, changed.length > 0 ? checkChanged(root, standards, changed) : [], changed.length);
|
|
236
|
-
}));
|
|
141
|
+
registerProofTools(server, { answer, readOnly, project });
|
|
237
142
|
await server.connect(new StdioServerTransport());
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
import { existsSync } from 'node:fs';
|
|
2
|
+
import { isAbsolute } from 'node:path';
|
|
3
|
+
import { z } from 'zod';
|
|
4
|
+
export function registerProofTools(server, { answer, readOnly, project }) {
|
|
5
|
+
server.registerTool('vouch_tests', {
|
|
6
|
+
title: 'Could these tests ever fail?',
|
|
7
|
+
description: "Reads test files and says, per test, whether a change to the code could break them: which compare no value, which leave an expectation unfinished, which compare something with itself, which catch and discard their own failure, and which never run. Where the checking happens in a helper it cannot follow it says so instead of accusing. Costs nothing and uses no model. Use it on tests that were just written - including tests you wrote yourself - before claiming a change is covered.",
|
|
8
|
+
inputSchema: {
|
|
9
|
+
project,
|
|
10
|
+
paths: z.array(z.string()).optional().describe('Paths inside the project to read, relative to it. Defaults to the whole project.'),
|
|
11
|
+
},
|
|
12
|
+
annotations: readOnly,
|
|
13
|
+
}, ({ project: root, paths }) => answer(async () => {
|
|
14
|
+
if (!isAbsolute(root) || !existsSync(root))
|
|
15
|
+
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
16
|
+
const { vouchReport } = await import('../engine/vouch/run-vouch.js');
|
|
17
|
+
return vouchReport(root, paths && paths.length > 0 ? paths : ['.']).report;
|
|
18
|
+
}));
|
|
19
|
+
server.registerTool('prove_tests_catch_bugs', {
|
|
20
|
+
title: 'Would the tests catch a bug here?',
|
|
21
|
+
description: "Breaks one source file in small, deliberate ways - a boundary moved by one, a condition inverted, an and that became an or, a returned flag flipped - runs the tests that reach that file, and reports every change none of them noticed. This is execution, not opinion: the line was covered, it just was not checked. Slow by nature (each change means a real test run, so expect minutes) and it needs a unit test framework. The file is always put back exactly as it was, and if the tests turn out not to load the file at all it says so instead of reporting false holes. Use it after writing or changing code, to find out whether the tests you are relying on would have stopped a mistake.",
|
|
22
|
+
inputSchema: {
|
|
23
|
+
project,
|
|
24
|
+
file: z.string().describe('One source file, relative to the project, e.g. src/pricing.js'),
|
|
25
|
+
limit: z.number().optional().describe('How many changes to try (default 6). More is a better answer and a longer wait.'),
|
|
26
|
+
budgetSeconds: z.number().optional().describe('Total seconds to spend before stopping and saying how many were left (default 600)'),
|
|
27
|
+
},
|
|
28
|
+
annotations: { readOnlyHint: false, destructiveHint: false, idempotentHint: true, openWorldHint: false },
|
|
29
|
+
}, ({ project: root, file, limit, budgetSeconds }) => answer(async () => {
|
|
30
|
+
if (!isAbsolute(root) || !existsSync(root))
|
|
31
|
+
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
32
|
+
// runProve prints its report; over MCP the report is the return value and printing would corrupt
|
|
33
|
+
// the protocol, so the pieces are called directly.
|
|
34
|
+
const { proveOne } = await import('../engine/prove/prove-command.js');
|
|
35
|
+
return proveOne(root, file, { limit, budgetSeconds });
|
|
36
|
+
}));
|
|
37
|
+
server.registerTool('is_the_work_done', {
|
|
38
|
+
title: 'Is this work actually done?',
|
|
39
|
+
description: "One verdict on a change, from the layers that each have their own measured accuracy: is anything failing, is the changed code tested by something that really reaches it, can the tests in the change fail at all, would they catch a bug (only when prove is asked for, because that costs minutes), and is what was asked for covered. Answers ready, not ready, or cannot say - and cannot say is never rounded up to ready. Use it before claiming a piece of work is finished, including your own: a green test run is not the same as work that is done.",
|
|
40
|
+
inputSchema: {
|
|
41
|
+
project,
|
|
42
|
+
diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
|
|
43
|
+
prove: z.boolean().optional().describe('Also break the changed code and run the tests against it. Costs minutes.'),
|
|
44
|
+
},
|
|
45
|
+
annotations: readOnly,
|
|
46
|
+
}, ({ project: root, diff, prove }) => answer(async () => {
|
|
47
|
+
if (!isAbsolute(root) || !existsSync(root))
|
|
48
|
+
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
49
|
+
// gather/decide/render are pure; runDone prints, which would corrupt the protocol.
|
|
50
|
+
const { gather } = await import('../engine/verdict/verdict-command.js');
|
|
51
|
+
const { decide, renderDecision } = await import('../engine/verdict/verdict.js');
|
|
52
|
+
const range = diff ?? 'main...HEAD';
|
|
53
|
+
const gathered = gather({ root, diff: range, prove });
|
|
54
|
+
if (!gathered)
|
|
55
|
+
return `Nothing changed in ${range}, or that range could not be read here.`;
|
|
56
|
+
return renderDecision(range, decide(gathered.evidence));
|
|
57
|
+
}));
|
|
58
|
+
server.registerTool('which_tests_to_run', {
|
|
59
|
+
title: 'Which tests could notice this change?',
|
|
60
|
+
description: "Narrows a test run to the tests that could notice a change, from facts in the source rather than a prediction: a test runs when it imports a changed file, reaches one through another module, is itself edited by the change, reads the repository at run time, or has failed recently. It refuses to narrow at all - and says why - when the change touches dependencies, build or test configuration, shared setup or fixtures, when the suite drives a browser, or when a changed file cannot be traced to any test. Use it to run the right tests quickly; what it leaves out, it leaves out for a stated reason.",
|
|
61
|
+
inputSchema: {
|
|
62
|
+
project,
|
|
63
|
+
diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
|
|
64
|
+
files: z.array(z.string()).optional().describe('The changed files, when they do not come from a git range'),
|
|
65
|
+
},
|
|
66
|
+
annotations: readOnly,
|
|
67
|
+
}, ({ project: root, diff, files }) => answer(async () => {
|
|
68
|
+
if (!isAbsolute(root) || !existsSync(root))
|
|
69
|
+
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
70
|
+
// choose/renderPick are pure; runPick prints, which would corrupt the protocol.
|
|
71
|
+
const { choose } = await import('../engine/pick/pick-command.js');
|
|
72
|
+
const { renderPick } = await import('../engine/pick/pick.js');
|
|
73
|
+
const range = diff ?? 'main...HEAD';
|
|
74
|
+
const chosen = choose({ root, diff: range, files });
|
|
75
|
+
if (!chosen)
|
|
76
|
+
return `Nothing changed in ${range}, or that range could not be read here.`;
|
|
77
|
+
return renderPick(chosen.picked, chosen.total, chosen.command, chosen.noHistory);
|
|
78
|
+
}));
|
|
79
|
+
server.registerTool('what_will_this_migration_do', {
|
|
80
|
+
title: 'What will this migration do to a live database?',
|
|
81
|
+
description: "Reads the migrations in a change and names what will happen when they run against a live database: a column dropped while the version still running keeps selecting it, a table dropped, an index built under a lock that blocks writes, a NOT NULL added to a populated table, a type change that rewrites it, a foreign key validated against every row, a data migration holding a transaction open, a step that cannot be undone. It also searches the repository for code that still uses the column being taken away. Reads SQL, Django, Rails and Alembic, judges only the forward path, and says plainly when it cannot read a migration. It does not know your table sizes, so it says what a statement does rather than how long it will take.",
|
|
82
|
+
inputSchema: {
|
|
83
|
+
project,
|
|
84
|
+
diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
|
|
85
|
+
files: z.array(z.string()).optional().describe('Migration files to read, when they do not come from a git range'),
|
|
86
|
+
},
|
|
87
|
+
annotations: readOnly,
|
|
88
|
+
}, ({ project: root, diff, files }) => answer(async () => {
|
|
89
|
+
if (!isAbsolute(root) || !existsSync(root))
|
|
90
|
+
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
91
|
+
const { readMigration, renderSchema, schemaFiles, stillUsed } = await import('../engine/schema/schema-command.js');
|
|
92
|
+
const cmd = { root, diff: diff ?? 'main...HEAD', files };
|
|
93
|
+
const migrations = schemaFiles(cmd).map((file) => readMigration(root, file));
|
|
94
|
+
if (migrations.length === 0)
|
|
95
|
+
return 'No migration or schema file in that change.';
|
|
96
|
+
return renderSchema(migrations, stillUsed(root, migrations.flatMap((m) => m.findings)));
|
|
97
|
+
}));
|
|
98
|
+
server.registerTool('project_standards', {
|
|
99
|
+
title: "What are this project's own rules?",
|
|
100
|
+
description: "The house rules this team has written down in .tewip/standards/, and whether a change keeps them. Returns two things kept carefully apart: rules Tewip checks and enforces (a forbid or require pattern over a set of files), and guidance it can only carry - prose for the parts no pattern captures, which is passed on when something is written but never reported as verified. Use it before writing or changing code in somebody's project, so what you produce matches their conventions rather than yours.",
|
|
101
|
+
inputSchema: {
|
|
102
|
+
project,
|
|
103
|
+
files: z.array(z.string()).optional().describe('Files to check against the rules. Omit to list what the project has written.'),
|
|
104
|
+
diff: z.string().optional().describe('A git range whose changed files should be checked instead'),
|
|
105
|
+
},
|
|
106
|
+
annotations: readOnly,
|
|
107
|
+
}, ({ project: root, files, diff }) => answer(async () => {
|
|
108
|
+
if (!isAbsolute(root) || !existsSync(root))
|
|
109
|
+
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
110
|
+
const { checkChanged, loadStandards, renderStandards } = await import('../engine/standards/standards-command.js');
|
|
111
|
+
const { changeSummary } = await import('../engine/quality/grounding.js');
|
|
112
|
+
const standards = loadStandards(root);
|
|
113
|
+
const changed = files && files.length > 0 ? files : diff ? (changeSummary(root, diff)?.files ?? []) : [];
|
|
114
|
+
return renderStandards(standards, changed.length > 0 ? checkChanged(root, standards, changed) : [], changed.length);
|
|
115
|
+
}));
|
|
116
|
+
}
|
package/package.json
CHANGED
|
@@ -1,81 +1,82 @@
|
|
|
1
|
-
{
|
|
2
|
-
"name": "tewip-ai",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
7
|
-
"
|
|
8
|
-
"
|
|
9
|
-
"
|
|
10
|
-
"
|
|
11
|
-
"
|
|
12
|
-
"
|
|
13
|
-
"
|
|
14
|
-
"
|
|
15
|
-
"
|
|
16
|
-
"
|
|
17
|
-
"
|
|
18
|
-
"
|
|
19
|
-
"
|
|
20
|
-
"
|
|
21
|
-
"
|
|
22
|
-
"
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
"
|
|
26
|
-
|
|
27
|
-
"
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
"
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
"
|
|
39
|
-
"
|
|
40
|
-
"
|
|
41
|
-
"
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
"test
|
|
46
|
-
"
|
|
47
|
-
"
|
|
48
|
-
"
|
|
49
|
-
"
|
|
50
|
-
"
|
|
51
|
-
"
|
|
52
|
-
"
|
|
53
|
-
"
|
|
54
|
-
"eval
|
|
55
|
-
"eval:
|
|
56
|
-
"eval:
|
|
57
|
-
"eval:
|
|
58
|
-
"eval:
|
|
59
|
-
"
|
|
60
|
-
"
|
|
61
|
-
"
|
|
62
|
-
"
|
|
63
|
-
"
|
|
64
|
-
"
|
|
65
|
-
"
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
"
|
|
69
|
-
|
|
70
|
-
"@
|
|
71
|
-
"
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
"
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
1
|
+
{
|
|
2
|
+
"name": "tewip-ai",
|
|
3
|
+
"version": "0.34.0",
|
|
4
|
+
"mcpName": "io.github.hmamut39/tewip",
|
|
5
|
+
"description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
|
|
6
|
+
"keywords": [
|
|
7
|
+
"test-failures",
|
|
8
|
+
"flaky-tests",
|
|
9
|
+
"triage",
|
|
10
|
+
"root-cause",
|
|
11
|
+
"ci",
|
|
12
|
+
"github-action",
|
|
13
|
+
"playwright",
|
|
14
|
+
"pytest",
|
|
15
|
+
"jest",
|
|
16
|
+
"vitest",
|
|
17
|
+
"junit",
|
|
18
|
+
"mcp",
|
|
19
|
+
"ai-agent",
|
|
20
|
+
"test-automation",
|
|
21
|
+
"qa",
|
|
22
|
+
"sdet",
|
|
23
|
+
"devops"
|
|
24
|
+
],
|
|
25
|
+
"homepage": "https://github.com/hmamut39/tewip#readme",
|
|
26
|
+
"repository": {
|
|
27
|
+
"type": "git",
|
|
28
|
+
"url": "https://github.com/hmamut39/tewip.git"
|
|
29
|
+
},
|
|
30
|
+
"type": "module",
|
|
31
|
+
"engines": {
|
|
32
|
+
"node": ">=22.18"
|
|
33
|
+
},
|
|
34
|
+
"bin": {
|
|
35
|
+
"tewip": "dist/cli/tewip.js"
|
|
36
|
+
},
|
|
37
|
+
"files": [
|
|
38
|
+
"LICENSE",
|
|
39
|
+
"QUICKSTART.md",
|
|
40
|
+
"README.md",
|
|
41
|
+
"TEAM-GUIDE.md",
|
|
42
|
+
"dist/"
|
|
43
|
+
],
|
|
44
|
+
"scripts": {
|
|
45
|
+
"test": "playwright test",
|
|
46
|
+
"test:loop": "node scripts/test-loop.ts 5",
|
|
47
|
+
"history": "node scripts/history.ts",
|
|
48
|
+
"diagnose": "node scripts/diagnose.ts",
|
|
49
|
+
"triage": "node scripts/triage.ts",
|
|
50
|
+
"explain": "node scripts/explain.ts",
|
|
51
|
+
"lab": "node scripts/lab.ts",
|
|
52
|
+
"test:engine": "node --test engine/test/*.test.ts outlets/test/*.test.ts mcp/test/*.test.ts",
|
|
53
|
+
"mcp": "node mcp/server.ts",
|
|
54
|
+
"eval": "node eval/evaluate.ts eval/ground-truth.jsonl --data data/real/gutenberg",
|
|
55
|
+
"eval:holdout": "node eval/evaluate.ts eval/ground-truth.holdout.jsonl --data data/real/gutenberg-holdout",
|
|
56
|
+
"eval:fresh": "node eval/evaluate.ts eval/ground-truth.fresh.jsonl --data data/real/gutenberg-fresh",
|
|
57
|
+
"eval:aug": "node eval/evaluate.ts eval/ground-truth.aug.jsonl --data data/real/gutenberg-aug",
|
|
58
|
+
"eval:aug2": "node eval/evaluate.ts eval/ground-truth.aug2.jsonl --data data/real/gutenberg-aug2",
|
|
59
|
+
"eval:supabase": "node eval/evaluate.ts eval/ground-truth.supabase.jsonl --data data/real/supabase",
|
|
60
|
+
"shadow": "node scripts/shadow.ts",
|
|
61
|
+
"harvest": "node eval/harvest/harvest.ts",
|
|
62
|
+
"eval:synthetic": "node eval/evaluate.ts eval/ground-truth.synthetic.jsonl",
|
|
63
|
+
"typecheck": "tsc --noEmit",
|
|
64
|
+
"build": "tsc -p tsconfig.build.json",
|
|
65
|
+
"prepack": "npm run build",
|
|
66
|
+
"tewip": "node cli/tewip.ts"
|
|
67
|
+
},
|
|
68
|
+
"license": "Apache-2.0",
|
|
69
|
+
"devDependencies": {
|
|
70
|
+
"@playwright/test": "^1.63.0",
|
|
71
|
+
"@types/node": "^26.5.1",
|
|
72
|
+
"typescript": "^7.0.2"
|
|
73
|
+
},
|
|
74
|
+
"dependencies": {
|
|
75
|
+
"@modelcontextprotocol/sdk": "^1.30.0",
|
|
76
|
+
"zod": "^4.6.5"
|
|
77
|
+
},
|
|
78
|
+
"bugs": {
|
|
79
|
+
"url": "https://github.com/hmamut39/tewip/issues"
|
|
80
|
+
},
|
|
81
|
+
"author": "Mamuti Aishan (Hasan)"
|
|
82
|
+
}
|