tewip-ai 0.11.0 → 0.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +29 -0
- package/TEAM-GUIDE.md +1 -0
- package/dist/cli/tewip.js +61 -0
- package/dist/engine/build/apply.js +99 -0
- package/dist/engine/build/build-command.js +245 -0
- package/dist/engine/build/build.js +89 -0
- package/dist/engine/build/propose-pr.js +88 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -181,6 +181,35 @@ Each reader was checked against real reports published in public repositories, w
|
|
|
181
181
|
defects were found before release - a Robot timestamp that is not a date, k6's numbers living under
|
|
182
182
|
`values`, and a JMeter line that read "mostly 200" when an assertion had failed.
|
|
183
183
|
|
|
184
|
+
## Build a feature, test first (new)
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
tewip build "naturalsize should take a sep argument that goes between the number and the unit"
|
|
188
|
+
tewip build --issue PROJ-123 # build from the ticket's own description and acceptance criteria
|
|
189
|
+
tewip build "..." --pr # commit the proved work on its own branch, open a draft PR
|
|
190
|
+
tewip build "..." --dry-run # prove it, show the patch, leave nothing on disk
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Tewip writes the **test first** and runs it. **If that test passes, it stops and keeps nothing** —
|
|
194
|
+
a test that passes before the feature exists is testing nothing. Then it writes the smallest code
|
|
195
|
+
that makes the test pass, and keeps the work only if the new test passes *and* nothing that was
|
|
196
|
+
passing has started to fail. Otherwise every file goes back byte for byte, including the test it
|
|
197
|
+
wrote. Nothing is ever committed.
|
|
198
|
+
|
|
199
|
+
Tried on two real public projects: it added a trailing-slash rule to **unjs/ufo** (4 lines,
|
|
200
|
+
TypeScript, $0.18) and a `sep` argument to **python-humanize** (2 lines, Python, $0.10), each with
|
|
201
|
+
its own failing-then-passing test and the project's whole suite still green. Asked for something
|
|
202
|
+
that already worked, it stopped after one model call and kept nothing.
|
|
203
|
+
|
|
204
|
+
It also refuses a feature that does not belong here at all. Handed a real Apache ticket about a
|
|
205
|
+
Flink HTTP sink, a URL library got an "HTTP sink" built into it - test passing, suite green, every
|
|
206
|
+
proof rule held, because when Tewip writes *both* the test and the code it can always make a
|
|
207
|
+
self-consistent pair pass. So before writing anything it now asks whether the words of the feature
|
|
208
|
+
mean anything in this repository, and refuses for free when they do not.
|
|
209
|
+
|
|
210
|
+
The old promise was *"it can't touch your code"*. The new one is *"it can't change your code without
|
|
211
|
+
showing you it works"*.
|
|
212
|
+
|
|
184
213
|
## Before anyone reads the diff (new)
|
|
185
214
|
|
|
186
215
|
```bash
|
package/TEAM-GUIDE.md
CHANGED
|
@@ -127,6 +127,7 @@ collects environment failures in one issue, and (optionally) opens a fix pull re
|
|
|
127
127
|
| `tewip spec "<feature>"` | An epic, its user stories and acceptance criteria, grounded in your code; files them in Jira with `--project PROJ --create` | Before the work is broken down |
|
|
128
128
|
| `tewip report report.xml --gate` | The CI command for GitLab, Jenkins, CircleCI, Azure, Bitbucket and the rest: diagnoses the run, comments on the GitLab merge request, and fails the build only on failures that need a person | Your CI is not GitHub Actions |
|
|
129
129
|
| `tewip watch --cmd "npm test"` | Runs your tests when you save a file and explains each failure straight away | While you are writing code |
|
|
130
|
+
| `tewip build "<what it should do>"` | Writes a feature test first, then the code, and keeps it only if the test failed without it and passes with it | You want a small feature built and proved |
|
|
130
131
|
| `tewip review --diff main...HEAD` | Which tests cover your change, which have flaked before, what nothing tests, and what to run first | Before you open a pull request |
|
|
131
132
|
| `tewip flaky --days 30` | Which tests to quarantine, which to fix, what they cost in CI minutes - and when a quarantined one can come back | Monday morning |
|
|
132
133
|
| `tewip note --days 7` | The weekly quality note for a lead, from runs already diagnosed (no model calls, free) | Monday morning |
|
package/dist/cli/tewip.js
CHANGED
|
@@ -67,6 +67,14 @@ Usage:
|
|
|
67
67
|
the GitLab merge request, tells Slack or Teams only when a
|
|
68
68
|
failure needs a person, and with --gate fails the build only on
|
|
69
69
|
those failures.
|
|
70
|
+
tewip build "<what it should do>" | --issue PROJ-123 [--dry-run] [--pr] [--out feature.patch]
|
|
71
|
+
Writes a feature, test first: the test must fail without the
|
|
72
|
+
code and pass with it, and every test that was passing must
|
|
73
|
+
still pass - otherwise everything goes back byte for byte.
|
|
74
|
+
--issue PROJ-123 Build from the ticket itself: its description and acceptance
|
|
75
|
+
criteria. Works with a public tracker without an account.
|
|
76
|
+
--pr Commit the proved work on a branch of its own and open a draft
|
|
77
|
+
pull request. Never merges, never commits to your branch.
|
|
70
78
|
tewip review [--diff main...HEAD] [--out review.md]
|
|
71
79
|
Before anyone reads the diff: which tests cover this change,
|
|
72
80
|
which of them have flaked before, what has gone wrong in these
|
|
@@ -256,6 +264,59 @@ async function main() {
|
|
|
256
264
|
process.exitCode = 1;
|
|
257
265
|
return;
|
|
258
266
|
}
|
|
267
|
+
case 'build': {
|
|
268
|
+
const typed = args.filter((a, i) => !a.startsWith('--') && !['--repairs', '--out', '--max-lines', '--issue'].includes(args[i - 1] ?? '')).join(' ').trim();
|
|
269
|
+
const issueKey = flag(args, 'issue');
|
|
270
|
+
let feature = typed;
|
|
271
|
+
let issueTitle = '';
|
|
272
|
+
if (issueKey) {
|
|
273
|
+
// A ticket says what was asked for better than a sentence typed twice.
|
|
274
|
+
const { Jira, jiraFromEnv, jiraHelp } = await import('../engine/work/jira.js');
|
|
275
|
+
const { featureFromIssue } = await import('../engine/work/jira-command.js');
|
|
276
|
+
const config = jiraFromEnv();
|
|
277
|
+
if (!config)
|
|
278
|
+
throw new Error(jiraHelp());
|
|
279
|
+
const issue = await new Jira(config).issue(issueKey);
|
|
280
|
+
console.error(`Read ${issue.key} (${issue.type}, ${issue.status})${issue.acceptanceCriteria ? ' with its acceptance criteria' : ''}.`);
|
|
281
|
+
issueTitle = `${issue.key}: ${issue.summary}`;
|
|
282
|
+
feature = [featureFromIssue(issue), typed].filter(Boolean).join('\n\n');
|
|
283
|
+
}
|
|
284
|
+
if (!feature)
|
|
285
|
+
throw new Error('usage: tewip build "<what the feature should do>" | --issue PROJ-123 [--dry-run] [--pr] [--out feature.patch]');
|
|
286
|
+
const { runBuild, renderBuild } = await import('../engine/build/build-command.js');
|
|
287
|
+
const dryRun = args.includes('--dry-run');
|
|
288
|
+
const result = await runBuild({
|
|
289
|
+
feature,
|
|
290
|
+
root: process.cwd(),
|
|
291
|
+
dryRun,
|
|
292
|
+
repairs: flag(args, 'repairs') === undefined ? 2 : Math.max(0, Number(flag(args, 'repairs')) || 0),
|
|
293
|
+
maxLines: flag(args, 'max-lines') ? Number(flag(args, 'max-lines')) : undefined,
|
|
294
|
+
out: flag(args, 'out'),
|
|
295
|
+
});
|
|
296
|
+
console.log(`
|
|
297
|
+
${renderBuild(result, issueTitle || feature, dryRun)}`);
|
|
298
|
+
if (result.status !== 'built') {
|
|
299
|
+
process.exitCode = 1;
|
|
300
|
+
return;
|
|
301
|
+
}
|
|
302
|
+
if (args.includes('--pr')) {
|
|
303
|
+
if (dryRun)
|
|
304
|
+
throw new Error('--pr and --dry-run ask for opposite things: one keeps the work, the other puts it back.');
|
|
305
|
+
const { proposePullRequest } = await import('../engine/build/propose-pr.js');
|
|
306
|
+
const proposed = proposePullRequest({
|
|
307
|
+
root: process.cwd(),
|
|
308
|
+
files: result.files,
|
|
309
|
+
title: (issueTitle || feature.split('\n')[0]).slice(0, 72),
|
|
310
|
+
body: result.why,
|
|
311
|
+
issue: issueKey,
|
|
312
|
+
});
|
|
313
|
+
console.log(`
|
|
314
|
+
${proposed.status === 'opened' ? 'Opened a draft pull request.' : proposed.status === 'committed' ? `Committed on ${proposed.branch}.` : 'No pull request was opened.'}`);
|
|
315
|
+
for (const line of proposed.next)
|
|
316
|
+
console.log(` ${line}`);
|
|
317
|
+
}
|
|
318
|
+
return;
|
|
319
|
+
}
|
|
259
320
|
case 'review': {
|
|
260
321
|
const { runReview } = await import('../engine/review/review-command.js');
|
|
261
322
|
await runReview({
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
// Applying a set of edits to a working tree, and being able to put everything back exactly as it
|
|
2
|
+
// was. Every step of building a feature is provisional until the tests say otherwise, so nothing
|
|
3
|
+
// here writes anything it cannot undo.
|
|
4
|
+
import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
|
5
|
+
import { dirname, relative, resolve, sep } from 'node:path';
|
|
6
|
+
import { mayChange } from '../autofix/code-target.js';
|
|
7
|
+
import { changedLines } from '../autofix/code-fix.js';
|
|
8
|
+
export class Workspace {
|
|
9
|
+
root;
|
|
10
|
+
touched = new Map();
|
|
11
|
+
constructor(root) {
|
|
12
|
+
this.root = resolve(root);
|
|
13
|
+
}
|
|
14
|
+
/** Remembers a file's contents the first time it is touched, so it can always be restored. */
|
|
15
|
+
remember(path) {
|
|
16
|
+
if (this.touched.has(path))
|
|
17
|
+
return;
|
|
18
|
+
this.touched.set(path, existsSync(path) ? readFileSync(path, 'utf8') : null);
|
|
19
|
+
}
|
|
20
|
+
write(path, content) {
|
|
21
|
+
this.remember(path);
|
|
22
|
+
mkdirSync(dirname(path), { recursive: true });
|
|
23
|
+
writeFileSync(path, content, 'utf8');
|
|
24
|
+
}
|
|
25
|
+
/** Everything back as it was: files restored, files Tewip created removed. */
|
|
26
|
+
restore() {
|
|
27
|
+
for (const [path, was] of this.touched) {
|
|
28
|
+
if (was === null)
|
|
29
|
+
rmSync(path, { force: true });
|
|
30
|
+
else
|
|
31
|
+
writeFileSync(path, was, 'utf8');
|
|
32
|
+
}
|
|
33
|
+
this.touched.clear();
|
|
34
|
+
}
|
|
35
|
+
/** Keeps the changes, and forgets how to undo them. */
|
|
36
|
+
keep() {
|
|
37
|
+
this.touched.clear();
|
|
38
|
+
}
|
|
39
|
+
/** What has been touched so far, for the report. */
|
|
40
|
+
files() {
|
|
41
|
+
return [...this.touched.keys()].map((p) => relative(this.root, p).split(sep).join('/'));
|
|
42
|
+
}
|
|
43
|
+
}
|
|
44
|
+
const CR = String.fromCharCode(13);
|
|
45
|
+
const NL = String.fromCharCode(10);
|
|
46
|
+
/**
|
|
47
|
+
* Applies a model's edits, refusing anything that is not allowed or does not match. Line endings
|
|
48
|
+
* are compared in one form, because a Windows checkout and a model's answer never agree on them.
|
|
49
|
+
*/
|
|
50
|
+
export function applyEdits(workspace, root, edits, maxLines) {
|
|
51
|
+
const applied = [];
|
|
52
|
+
const refused = [];
|
|
53
|
+
for (const edit of edits) {
|
|
54
|
+
const path = resolve(root, edit.file);
|
|
55
|
+
const rel = relative(resolve(root), path).split(sep).join('/');
|
|
56
|
+
if (rel.startsWith('..') || resolve(path).startsWith('..')) {
|
|
57
|
+
refused.push(`${edit.file} is outside the project`);
|
|
58
|
+
continue;
|
|
59
|
+
}
|
|
60
|
+
const allowed = mayChange(path);
|
|
61
|
+
if (!allowed.allowed) {
|
|
62
|
+
refused.push(`${rel} may not be changed: ${allowed.why}`);
|
|
63
|
+
continue;
|
|
64
|
+
}
|
|
65
|
+
const exists = existsSync(path);
|
|
66
|
+
if (!exists) {
|
|
67
|
+
if (edit.before.trim()) {
|
|
68
|
+
refused.push(`${rel} does not exist, but the change expects text to already be in it`);
|
|
69
|
+
continue;
|
|
70
|
+
}
|
|
71
|
+
workspace.write(path, edit.after);
|
|
72
|
+
applied.push({ ...edit, path, was: null, changed: edit.after.split(NL).length });
|
|
73
|
+
continue;
|
|
74
|
+
}
|
|
75
|
+
const raw = readFileSync(path, 'utf8');
|
|
76
|
+
const crlf = raw.includes(CR + NL);
|
|
77
|
+
const original = crlf ? raw.split(CR + NL).join(NL) : raw;
|
|
78
|
+
const before = edit.before.split(CR + NL).join(NL);
|
|
79
|
+
const after = edit.after.split(CR + NL).join(NL);
|
|
80
|
+
if (!before.trim()) {
|
|
81
|
+
refused.push(`${rel} already exists, so a change to it must say which text to replace`);
|
|
82
|
+
continue;
|
|
83
|
+
}
|
|
84
|
+
const occurrences = original.split(before).length - 1;
|
|
85
|
+
if (occurrences !== 1) {
|
|
86
|
+
refused.push(`the text to replace in ${rel} ${occurrences === 0 ? 'is not in the file' : `appears ${occurrences} times`}`);
|
|
87
|
+
continue;
|
|
88
|
+
}
|
|
89
|
+
const changed = original.replace(before, after);
|
|
90
|
+
const touched = changedLines(original, changed);
|
|
91
|
+
if (touched > maxLines) {
|
|
92
|
+
refused.push(`the change to ${rel} touches ${touched} lines, which is a rewrite rather than a feature`);
|
|
93
|
+
continue;
|
|
94
|
+
}
|
|
95
|
+
workspace.write(path, crlf ? changed.split(NL).join(CR + NL) : changed);
|
|
96
|
+
applied.push({ ...edit, path, was: raw, changed: touched });
|
|
97
|
+
}
|
|
98
|
+
return { applied, refused };
|
|
99
|
+
}
|
|
@@ -0,0 +1,245 @@
|
|
|
1
|
+
// `tewip build "<what it should do>"`: a feature, written test first and kept only when proved.
|
|
2
|
+
//
|
|
3
|
+
// The order is the product. Tewip writes the test, runs it, and **requires it to fail** - a test
|
|
4
|
+
// that passes before the feature exists is testing nothing, and that single check is what stops a
|
|
5
|
+
// model from writing code that agrees with itself. Then it writes the code, and keeps it only when
|
|
6
|
+
// the new test passes and nothing that was passing has started to fail.
|
|
7
|
+
//
|
|
8
|
+
// Nothing is committed. Nothing is merged. What cannot be proved is put back byte for byte.
|
|
9
|
+
import { existsSync, readFileSync, writeFileSync, mkdirSync } from 'node:fs';
|
|
10
|
+
import { basename, dirname, join, relative, resolve, sep } from 'node:path';
|
|
11
|
+
import { spawnSync } from 'node:child_process';
|
|
12
|
+
import { chooseFramework, detectFrameworks } from '../quality/frameworks.js';
|
|
13
|
+
import { existingTests, relatedCodeSources, relatedTestSources } from '../quality/grounding.js';
|
|
14
|
+
import { writerFromEnv } from '../quality/model.js';
|
|
15
|
+
import { usageLine } from '../llm/usage.js';
|
|
16
|
+
import { failingTests } from '../autofix/code-fix.js';
|
|
17
|
+
import { lineDiff, renderDiff } from '../autofix/line-diff.js';
|
|
18
|
+
import { sourceFiles } from '../secure/secrets.js';
|
|
19
|
+
import { applyEdits, Workspace } from './apply.js';
|
|
20
|
+
import { repairFeatureCode, writeFeatureCode, writeFeatureTest } from './build.js';
|
|
21
|
+
const run = (command, cwd, env, timeoutMs = 10 * 60_000) => {
|
|
22
|
+
const r = spawnSync(command, { cwd, env: { ...process.env, ...env }, shell: true, encoding: 'utf8', timeout: timeoutMs, maxBuffer: 20 * 1024 * 1024 });
|
|
23
|
+
const output = `${r.stdout ?? ''}\n${r.stderr ?? ''}${r.error ? `\n${r.error.message}` : ''}`.replace(/\u001b\[[0-9;]*[A-Za-z]/g, '').trim();
|
|
24
|
+
return { passed: r.status === 0, output: output.slice(-6000) };
|
|
25
|
+
};
|
|
26
|
+
const CODE_FILE = /\.(?:[cm]?[jt]sx?|py|rb|go|java|kt|cs|php|rs|vue|svelte)$/i;
|
|
27
|
+
/**
|
|
28
|
+
* The files that define the things the feature names. Asking for "naturalsize should take a sep
|
|
29
|
+
* argument" and handing the model number.py and time.py - which is what similarity alone did -
|
|
30
|
+
* makes it answer, correctly, that it cannot see the code. The file that defines `naturalsize` is
|
|
31
|
+
* the one it needs, and finding it is a search, not a guess.
|
|
32
|
+
*/
|
|
33
|
+
export function definingSources(root, feature, limit = 3) {
|
|
34
|
+
const names = [...new Set((feature.match(/[A-Za-z_][A-Za-z0-9_]{3,}/g) ?? []).filter((w) => /[A-Z_]/.test(w) || w.length > 6))].slice(0, 12);
|
|
35
|
+
if (names.length === 0)
|
|
36
|
+
return [];
|
|
37
|
+
const scored = [];
|
|
38
|
+
for (const file of sourceFiles(root, 3000)) {
|
|
39
|
+
if (!CODE_FILE.test(file) || /(?:^|[\\/])(?:test|tests|spec|__tests__)[\\/]/i.test(file))
|
|
40
|
+
continue;
|
|
41
|
+
let text;
|
|
42
|
+
try {
|
|
43
|
+
text = readFileSync(file, 'utf8');
|
|
44
|
+
}
|
|
45
|
+
catch {
|
|
46
|
+
continue;
|
|
47
|
+
}
|
|
48
|
+
// String.raw, because inside a plain template literal "\s" is just "s" and "\b" is a backspace.
|
|
49
|
+
const defines = new RegExp(String.raw `(?:^\s*(?:async\s+)?def\s+|export\s+(?:async\s+)?(?:function|const|class)\s+|function\s+)`, 'm');
|
|
50
|
+
const score = names.filter((n) => new RegExp(`${defines.source}${n}\\b`, 'm').test(text)).length;
|
|
51
|
+
if (score > 0)
|
|
52
|
+
scored.push({ file, score });
|
|
53
|
+
}
|
|
54
|
+
return scored.sort((a, b) => b.score - a.score).slice(0, limit).map((s) => s.file);
|
|
55
|
+
}
|
|
56
|
+
/**
|
|
57
|
+
* Does this feature belong in this project at all?
|
|
58
|
+
*
|
|
59
|
+
* Found by trying it: a real Apache ticket about a Flink HTTP sink was handed to a URL library, and
|
|
60
|
+
* Tewip built an "HTTP sink" into it - 28 lines, its own test passing, the suite green. Every proof
|
|
61
|
+
* rule held, because when Tewip writes both the test and the code it can always make a
|
|
62
|
+
* self-consistent pair pass. Nothing else referenced the invented API, so nothing else could break.
|
|
63
|
+
*
|
|
64
|
+
* The missing check is the obvious one: the words of the feature have to mean something here. A
|
|
65
|
+
* project's own vocabulary is its file names and what it exports, and a feature that shares almost
|
|
66
|
+
* none of it is a feature for some other repository.
|
|
67
|
+
*/
|
|
68
|
+
export function belongsHere(root, feature, sources, testNames) {
|
|
69
|
+
const stop = new Set(['should', 'when', 'with', 'that', 'this', 'from', 'into', 'have', 'been', 'will', 'must', 'also', 'able', 'given', 'then', 'there', 'their', 'which', 'while', 'where', 'after', 'before', 'value', 'values', 'return', 'returns', 'support', 'supports', 'accept', 'accepts', 'option', 'options', 'default', 'behaviour', 'behavior', 'feature', 'example', 'instead', 'without']);
|
|
70
|
+
const words = (text) => new Set((text.toLowerCase().match(/[a-z][a-z0-9]{3,}/g) ?? []).filter((w) => !stop.has(w)));
|
|
71
|
+
const asked = [...words(feature)];
|
|
72
|
+
if (asked.length < 3)
|
|
73
|
+
return { ok: true, why: '' };
|
|
74
|
+
const vocabulary = new Set();
|
|
75
|
+
for (const s of sources)
|
|
76
|
+
for (const w of words(`${s.file} ${s.content.slice(0, 20_000)}`))
|
|
77
|
+
vocabulary.add(w);
|
|
78
|
+
for (const name of testNames)
|
|
79
|
+
for (const w of words(name))
|
|
80
|
+
vocabulary.add(w);
|
|
81
|
+
// The names of a project's files are not its vocabulary: "naturalsize" lives inside filesize.py,
|
|
82
|
+
// not in its name, and judging by names alone refused a feature that really did belong here.
|
|
83
|
+
let read = 0;
|
|
84
|
+
for (const file of sourceFiles(root, 1200)) {
|
|
85
|
+
if (!CODE_FILE.test(file) || read >= 60)
|
|
86
|
+
continue;
|
|
87
|
+
read++;
|
|
88
|
+
try {
|
|
89
|
+
for (const w of words(`${basename(file)} ${readFileSync(file, 'utf8').slice(0, 20_000)}`))
|
|
90
|
+
vocabulary.add(w);
|
|
91
|
+
}
|
|
92
|
+
catch {
|
|
93
|
+
// unreadable here is simply one less file of vocabulary
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
if (vocabulary.size === 0)
|
|
97
|
+
return { ok: true, why: '' };
|
|
98
|
+
const shared = asked.filter((w) => vocabulary.has(w));
|
|
99
|
+
const share = shared.length / asked.length;
|
|
100
|
+
if (share >= 0.2)
|
|
101
|
+
return { ok: true, why: '' };
|
|
102
|
+
return {
|
|
103
|
+
ok: false,
|
|
104
|
+
why: `almost nothing in this project mentions what that describes (${shared.length} of ${asked.length} of its words appear here${shared.length ? `: ${shared.slice(0, 5).join(', ')}` : ''}). If this is the right repository, say the feature in its own words - the names of the functions and files it should change. Tewip can make a test of its own invention pass in any project, and that would prove nothing.`,
|
|
105
|
+
};
|
|
106
|
+
}
|
|
107
|
+
/** A test file name in the project's own shape, that does not collide with an existing one. */
|
|
108
|
+
function testPathFor(framework, feature) {
|
|
109
|
+
const stem = feature.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '').split('-').slice(0, 4).join('-') || 'feature';
|
|
110
|
+
let path = join(framework.testDir, framework.fileFor(stem));
|
|
111
|
+
let n = 2;
|
|
112
|
+
while (existsSync(path))
|
|
113
|
+
path = join(framework.testDir, framework.fileFor(`${stem}-${n++}`));
|
|
114
|
+
return path;
|
|
115
|
+
}
|
|
116
|
+
/** What the model is shown: the files that define what the feature names, then the similar ones. */
|
|
117
|
+
function sourcesFor(root, feature) {
|
|
118
|
+
const defining = definingSources(root, feature).map((file) => ({ file: relative(root, file).split(sep).join('/'), content: readFileSync(file, 'utf8') }));
|
|
119
|
+
const similar = relatedCodeSources(root, feature).map((s) => ({ file: s.file, content: s.content }));
|
|
120
|
+
const seen = new Set(defining.map((s) => s.file));
|
|
121
|
+
return [...defining, ...similar.filter((s) => !seen.has(s.file))].slice(0, 4);
|
|
122
|
+
}
|
|
123
|
+
export async function runBuild(cmd) {
|
|
124
|
+
const root = resolve(cmd.root);
|
|
125
|
+
const model = cmd.model ?? writerFromEnv(root);
|
|
126
|
+
if (!model)
|
|
127
|
+
throw new Error('building a feature needs a model. Set TEWIP_API_KEY (and TEWIP_PROVIDER), or OPENAI_API_KEY.');
|
|
128
|
+
const runIt = cmd.runCommand ?? ((command, cwd) => run(command, cwd, framework?.env));
|
|
129
|
+
const framework = chooseFramework(detectFrameworks(root), undefined, false) ?? detectFrameworks(root)[0] ?? null;
|
|
130
|
+
if (!framework) {
|
|
131
|
+
return { status: 'no-framework', why: 'no test framework was found here, and a feature Tewip cannot test is a feature it will not write.', files: [], diff: '', log: [] };
|
|
132
|
+
}
|
|
133
|
+
const tests = existingTests(root);
|
|
134
|
+
const ctx = {
|
|
135
|
+
feature: cmd.feature,
|
|
136
|
+
framework,
|
|
137
|
+
exampleTest: framework.example ? readFileSync(framework.example, 'utf8') : (relatedTestSources(root, cmd.feature)[0]?.content ?? ''),
|
|
138
|
+
sources: sourcesFor(root, cmd.feature),
|
|
139
|
+
testNames: tests.tests,
|
|
140
|
+
};
|
|
141
|
+
const log = [`framework: ${framework.label}`, `source files read: ${ctx.sources.map((s) => s.file).join(', ') || 'none found'}`];
|
|
142
|
+
// Before anything is written: is this even a feature for this repository?
|
|
143
|
+
const belongs = belongsHere(root, cmd.feature, ctx.sources, ctx.testNames);
|
|
144
|
+
if (!belongs.ok)
|
|
145
|
+
return { status: 'not-this-project', why: belongs.why, files: [], diff: '', log: [...log, belongs.why] };
|
|
146
|
+
const workspace = new Workspace(root);
|
|
147
|
+
const maxLines = cmd.maxLines ?? 120;
|
|
148
|
+
const stop = (status, why) => {
|
|
149
|
+
workspace.restore();
|
|
150
|
+
return { status, why, files: [], diff: '', log: [...log, why] };
|
|
151
|
+
};
|
|
152
|
+
// 1. The test, first - and it has to fail.
|
|
153
|
+
console.log('Writing the test first...');
|
|
154
|
+
const draft = await writeFeatureTest(model, ctx);
|
|
155
|
+
const testPath = testPathFor(framework, cmd.feature);
|
|
156
|
+
mkdirSync(dirname(testPath), { recursive: true });
|
|
157
|
+
workspace.write(testPath, draft.code);
|
|
158
|
+
const testRel = relative(root, testPath).split(sep).join('/');
|
|
159
|
+
log.push(`wrote ${testRel}: ${draft.checks}`);
|
|
160
|
+
const before = runIt(framework.command(testPath), framework.cwd);
|
|
161
|
+
if (before.passed) {
|
|
162
|
+
return stop('test-passes-already', `the test passes without any new code, so it is not testing the feature. Either ${cmd.feature} already works, or the test checks something that is already true. Nothing was kept.`);
|
|
163
|
+
}
|
|
164
|
+
log.push(`the test fails today, as it must: ${draft.failsBecause}`);
|
|
165
|
+
// What is already broken here, so the feature is never blamed for it.
|
|
166
|
+
const baselineRun = runIt(framework.suite, framework.cwd);
|
|
167
|
+
const alreadyFailing = new Set(baselineRun.passed ? [] : failingTests(baselineRun.output));
|
|
168
|
+
if (alreadyFailing.size > 0)
|
|
169
|
+
log.push(`${alreadyFailing.size} test(s) were already failing before anything was written, and are not counted against the feature`);
|
|
170
|
+
// 2. The code.
|
|
171
|
+
console.log('Writing the code...');
|
|
172
|
+
let change = await writeFeatureCode(model, ctx, { file: testRel, code: draft.code }, before.output);
|
|
173
|
+
const attempts = Math.max(1, (cmd.repairs ?? 2) + 1);
|
|
174
|
+
for (let attempt = 1; attempt <= attempts; attempt++) {
|
|
175
|
+
if (change.outcome === 'cannot' || change.edits.length === 0) {
|
|
176
|
+
return stop('no-code', `Tewip did not write code for this: ${change.why || 'it could not see how, from what this project gave it'}. The test it wrote is a good description of what is missing; run \`tewip build\` again with more detail, or write the code yourself against that test.`);
|
|
177
|
+
}
|
|
178
|
+
const { applied, refused } = applyEdits(workspace, root, change.edits, maxLines);
|
|
179
|
+
for (const why of refused)
|
|
180
|
+
log.push(`refused: ${why}`);
|
|
181
|
+
if (applied.length === 0) {
|
|
182
|
+
return stop('refused', `every change offered was refused: ${refused.join('; ') || 'nothing could be applied'}`);
|
|
183
|
+
}
|
|
184
|
+
log.push(`attempt ${attempt}: changed ${applied.map((e) => `${relative(root, e.path).split(sep).join('/')} (${e.changed} line(s))`).join(', ')}`);
|
|
185
|
+
const after = runIt(framework.command(testPath), framework.cwd);
|
|
186
|
+
if (!after.passed) {
|
|
187
|
+
if (attempt === attempts)
|
|
188
|
+
return stop('still-failing', 'the test still fails after every attempt, so everything was put back byte for byte.');
|
|
189
|
+
console.log(`The test still fails; correcting (attempt ${attempt + 1})...`);
|
|
190
|
+
change = await repairFeatureCode(model, ctx, { edits: change.edits, output: after.output, broke: [] });
|
|
191
|
+
continue;
|
|
192
|
+
}
|
|
193
|
+
const suite = runIt(framework.suite, framework.cwd);
|
|
194
|
+
const broke = failingTests(suite.output).filter((name) => !alreadyFailing.has(name));
|
|
195
|
+
if (!suite.passed && (broke.length > 0 || failingTests(suite.output).length === 0)) {
|
|
196
|
+
if (attempt === attempts)
|
|
197
|
+
return stop('broke-others', `the new test passes, but this broke tests that were passing: ${broke.slice(0, 6).join(', ') || 'see the suite output'}. Everything was put back.`);
|
|
198
|
+
console.log('The new test passes but something else broke; correcting...');
|
|
199
|
+
change = await repairFeatureCode(model, ctx, { edits: change.edits, output: suite.output, broke });
|
|
200
|
+
continue;
|
|
201
|
+
}
|
|
202
|
+
// Proved: the new test passes, and nothing that was passing fails.
|
|
203
|
+
log.push('the new test passes, and everything that was passing still passes');
|
|
204
|
+
const diff = [
|
|
205
|
+
renderDiff(testRel, lineDiff('', draft.code)),
|
|
206
|
+
...applied.map((e) => renderDiff(relative(root, e.path).split(sep).join('/'), lineDiff(e.was ?? '', readFileSync(e.path, 'utf8')))),
|
|
207
|
+
]
|
|
208
|
+
.filter(Boolean)
|
|
209
|
+
.join('\n');
|
|
210
|
+
const files = [testRel, ...applied.map((e) => relative(root, e.path).split(sep).join('/'))];
|
|
211
|
+
if (cmd.dryRun) {
|
|
212
|
+
workspace.restore();
|
|
213
|
+
log.push('dry run: everything was put back, and only the patch above is yours to apply');
|
|
214
|
+
}
|
|
215
|
+
else {
|
|
216
|
+
workspace.keep();
|
|
217
|
+
}
|
|
218
|
+
if (cmd.out) {
|
|
219
|
+
mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
|
|
220
|
+
writeFileSync(resolve(cmd.out), `${diff}\n`, 'utf8');
|
|
221
|
+
}
|
|
222
|
+
return { status: 'built', why: change.why, testFile: testRel, files, diff, log };
|
|
223
|
+
}
|
|
224
|
+
return stop('still-failing', 'no change could be proved');
|
|
225
|
+
}
|
|
226
|
+
/** What the person reads afterwards. */
|
|
227
|
+
export function renderBuild(result, feature, dryRun) {
|
|
228
|
+
const lines = [`# ${result.status === 'built' ? 'Built' : 'Not built'}: ${feature}`, ''];
|
|
229
|
+
for (const line of result.log)
|
|
230
|
+
lines.push(`- ${line}`);
|
|
231
|
+
lines.push('');
|
|
232
|
+
if (result.status !== 'built') {
|
|
233
|
+
lines.push(result.why, '', 'Nothing was kept. Tewip only leaves code behind when the test it wrote first fails without it and passes with it.');
|
|
234
|
+
return lines.join('\n');
|
|
235
|
+
}
|
|
236
|
+
lines.push(`**What it does.** ${result.why}`, '');
|
|
237
|
+
lines.push('```diff', result.diff, '```', '');
|
|
238
|
+
lines.push(dryRun ? 'This was a dry run: nothing is on disk. Apply the patch above if you want it.' : `Left in your working tree: ${result.files.join(', ')}. Nothing was committed.`);
|
|
239
|
+
lines.push('');
|
|
240
|
+
lines.push('It is proved in the only way that means anything: the test failed without this code and passes with it, and every test that was passing still passes. Read it before you commit it.');
|
|
241
|
+
const usage = usageLine();
|
|
242
|
+
if (usage)
|
|
243
|
+
lines.push('', usage);
|
|
244
|
+
return lines.join('\n');
|
|
245
|
+
}
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
export const BUILD_PROMPT_VERSION = 'build-1';
|
|
2
|
+
const RULES = `Rules that do not bend:
|
|
3
|
+
- Use only what the project already has: its framework, its style, its helpers, its way of importing. Copy the shape of the example test and the example source given.
|
|
4
|
+
- Never weaken or delete an existing test, and never change what one asserts.
|
|
5
|
+
- Keep the change small and local. Add a file when the project's own layout calls for one; do not reorganise anything.
|
|
6
|
+
- Write no comments that restate the code, no logging, no defensive try/catch around things that cannot fail.
|
|
7
|
+
- before must be copied character for character from the file given and appear in it exactly once. For a new file, before is empty and after is the whole file.`;
|
|
8
|
+
const TEST_SYSTEM = `You are Tewip, writing the test for a feature that does not exist yet.
|
|
9
|
+
|
|
10
|
+
Write ONE test that checks the described behaviour, in the project's own framework and style. It must fail when run now, and fail for the right reason: the behaviour is missing, not because of a typo, a missing import or a file that does not exist. Use only functions, files and exports that already exist, or the one new export the feature obviously needs - name it as the project would.
|
|
11
|
+
|
|
12
|
+
checks is one sentence naming the behaviour the test asserts ("a full URL keeps its query when the trailing slash is removed"). It is not advice about file names or where to put things.
|
|
13
|
+
failsBecause is the behaviour that is missing today, in one sentence.
|
|
14
|
+
|
|
15
|
+
${RULES}`;
|
|
16
|
+
const CODE_SYSTEM = `You are Tewip, writing the code that makes a failing test pass.
|
|
17
|
+
|
|
18
|
+
You are given the feature, the test, the real error it produced, and the project's relevant source. Write the smallest change that makes that test pass because the behaviour now exists - never by weakening the test, never by special-casing the test's own values.
|
|
19
|
+
|
|
20
|
+
If the feature cannot be built from what you were given - the code you need is not here, or the request is ambiguous in a way that changes the answer - set outcome to "cannot" and return no edits. Saying "I cannot" is a correct answer; a guess that happens to pass one test is not.
|
|
21
|
+
|
|
22
|
+
${RULES}`;
|
|
23
|
+
const REPAIR_SYSTEM = `You are Tewip, fixing code you just wrote. The test still fails, or other tests broke. You are given what you changed and the real output.
|
|
24
|
+
|
|
25
|
+
Change your own code so the failing test passes and everything that was passing still passes. Never change the test. If you cannot, set outcome to "cannot".
|
|
26
|
+
|
|
27
|
+
${RULES}`;
|
|
28
|
+
const TEST_SCHEMA = {
|
|
29
|
+
type: 'object',
|
|
30
|
+
additionalProperties: false,
|
|
31
|
+
required: ['code', 'checks', 'failsBecause'],
|
|
32
|
+
properties: { code: { type: 'string' }, checks: { type: 'string' }, failsBecause: { type: 'string' } },
|
|
33
|
+
};
|
|
34
|
+
const EDIT = {
|
|
35
|
+
type: 'object',
|
|
36
|
+
additionalProperties: false,
|
|
37
|
+
required: ['file', 'before', 'after'],
|
|
38
|
+
properties: { file: { type: 'string' }, before: { type: 'string' }, after: { type: 'string' } },
|
|
39
|
+
};
|
|
40
|
+
const CODE_SCHEMA = {
|
|
41
|
+
type: 'object',
|
|
42
|
+
additionalProperties: false,
|
|
43
|
+
required: ['outcome', 'why', 'edits'],
|
|
44
|
+
properties: { outcome: { type: 'string', enum: ['built', 'cannot'] }, why: { type: 'string' }, edits: { type: 'array', items: EDIT } },
|
|
45
|
+
};
|
|
46
|
+
export async function writeFeatureTest(model, ctx) {
|
|
47
|
+
const raw = await model.complete({
|
|
48
|
+
system: TEST_SYSTEM,
|
|
49
|
+
user: JSON.stringify({
|
|
50
|
+
feature: ctx.feature,
|
|
51
|
+
framework: ctx.framework.label,
|
|
52
|
+
exampleTest: ctx.exampleTest.slice(0, 6000),
|
|
53
|
+
existingTestNames: ctx.testNames.slice(0, 60),
|
|
54
|
+
source: ctx.sources.map((s) => ({ file: s.file, content: s.content.slice(0, 6000) })),
|
|
55
|
+
}),
|
|
56
|
+
schemaName: 'tewip_feature_test',
|
|
57
|
+
schema: TEST_SCHEMA,
|
|
58
|
+
});
|
|
59
|
+
return JSON.parse(raw);
|
|
60
|
+
}
|
|
61
|
+
export async function writeFeatureCode(model, ctx, test, error) {
|
|
62
|
+
const raw = await model.complete({
|
|
63
|
+
system: CODE_SYSTEM,
|
|
64
|
+
user: JSON.stringify({
|
|
65
|
+
feature: ctx.feature,
|
|
66
|
+
test: { file: test.file, code: test.code.slice(0, 8000) },
|
|
67
|
+
errorItProduced: error.slice(0, 4000),
|
|
68
|
+
source: ctx.sources.map((s) => ({ file: s.file, content: s.content.slice(0, 12_000) })),
|
|
69
|
+
}),
|
|
70
|
+
schemaName: 'tewip_feature_code',
|
|
71
|
+
schema: CODE_SCHEMA,
|
|
72
|
+
});
|
|
73
|
+
return JSON.parse(raw);
|
|
74
|
+
}
|
|
75
|
+
export async function repairFeatureCode(model, ctx, attempt) {
|
|
76
|
+
const raw = await model.complete({
|
|
77
|
+
system: REPAIR_SYSTEM,
|
|
78
|
+
user: JSON.stringify({
|
|
79
|
+
feature: ctx.feature,
|
|
80
|
+
whatYouChanged: attempt.edits.map((e) => ({ file: e.file, after: e.after.slice(0, 4000) })),
|
|
81
|
+
output: attempt.output.slice(0, 4000),
|
|
82
|
+
testsThatBroke: attempt.broke.slice(0, 10),
|
|
83
|
+
source: ctx.sources.map((s) => ({ file: s.file, content: s.content.slice(0, 12_000) })),
|
|
84
|
+
}),
|
|
85
|
+
schemaName: 'tewip_feature_repair',
|
|
86
|
+
schema: CODE_SCHEMA,
|
|
87
|
+
});
|
|
88
|
+
return JSON.parse(raw);
|
|
89
|
+
}
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
// Turning proved work into a pull request - the last step of the loop, and the one where Tewip
|
|
2
|
+
// touches somebody's repository, so it is the most careful.
|
|
3
|
+
//
|
|
4
|
+
// Three rules, enforced here rather than promised:
|
|
5
|
+
// it only ever pushes a new branch of its own, never yours;
|
|
6
|
+
// it never merges, and opens the request as a draft so nobody can merge it by reflex;
|
|
7
|
+
// it describes exactly what was proved and what was not, because a pull request nobody can check
|
|
8
|
+
// is worse than a patch file.
|
|
9
|
+
//
|
|
10
|
+
// The work must already be proved before anything reaches this file. Nothing here runs a test or
|
|
11
|
+
// decides anything; it commits what was verified and opens the request.
|
|
12
|
+
import { spawnSync } from 'node:child_process';
|
|
13
|
+
const run = (cwd, command, args) => {
|
|
14
|
+
const r = spawnSync(command, args, { cwd, encoding: 'utf8', timeout: 120_000, maxBuffer: 16 * 1024 * 1024 });
|
|
15
|
+
return { ok: r.status === 0, out: `${r.stdout ?? ''}${r.stderr ?? ''}`.trim() };
|
|
16
|
+
};
|
|
17
|
+
/** A branch name a person can read at a glance, and that cannot collide with an existing one. */
|
|
18
|
+
export function branchNameFor(title, now = new Date()) {
|
|
19
|
+
const stem = title.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '').split('-').slice(0, 6).join('-') || 'change';
|
|
20
|
+
return `tewip/${stem}-${now.toISOString().slice(2, 16).replace(/\D/g, '')}`;
|
|
21
|
+
}
|
|
22
|
+
/** The pull request body: what was built, and precisely how far the proof goes. */
|
|
23
|
+
export function requestBody(body, files, issue) {
|
|
24
|
+
return [
|
|
25
|
+
body,
|
|
26
|
+
'',
|
|
27
|
+
`Files: ${files.map((f) => `\`${f}\``).join(', ')}`,
|
|
28
|
+
issue ? `From: ${issue}` : '',
|
|
29
|
+
'',
|
|
30
|
+
'---',
|
|
31
|
+
'',
|
|
32
|
+
'Written by Tewip and kept only because it was proved: the test failed without this change and',
|
|
33
|
+
'passes with it, and every test that was passing still passes.',
|
|
34
|
+
'',
|
|
35
|
+
'What that does **not** say: whether this is the design you want, whether it is the right place',
|
|
36
|
+
'for it, or whether it is complete. That is what your review is for. This is a draft on a branch',
|
|
37
|
+
'of its own; Tewip never merges anything.',
|
|
38
|
+
]
|
|
39
|
+
.filter((line) => line !== '')
|
|
40
|
+
.join('\n');
|
|
41
|
+
}
|
|
42
|
+
/**
|
|
43
|
+
* Commits the proved files on a fresh branch and opens a draft pull request with the `gh` CLI the
|
|
44
|
+
* developer is already logged into. When there is no `gh`, the branch is still pushed and the
|
|
45
|
+
* command to open the request by hand is printed - half a job done is better than a silent failure.
|
|
46
|
+
*/
|
|
47
|
+
export function proposePullRequest(input) {
|
|
48
|
+
const { root, files } = input;
|
|
49
|
+
if (files.length === 0)
|
|
50
|
+
return { status: 'refused', next: ['there was nothing to commit'] };
|
|
51
|
+
if (!run(root, 'git', ['rev-parse', '--git-dir']).ok)
|
|
52
|
+
return { status: 'refused', next: ['this is not a git checkout, so there is no branch to open a request from'] };
|
|
53
|
+
const branch = branchNameFor(input.title);
|
|
54
|
+
const base = run(root, 'git', ['rev-parse', '--abbrev-ref', 'HEAD']).out.trim() || 'main';
|
|
55
|
+
// A branch of Tewip's own, from wherever the person is standing. Their branch is never committed to.
|
|
56
|
+
const made = run(root, 'git', ['checkout', '-b', branch]);
|
|
57
|
+
if (!made.ok)
|
|
58
|
+
return { status: 'refused', next: [`could not create the branch ${branch}: ${made.out.slice(0, 200)}`] };
|
|
59
|
+
const added = run(root, 'git', ['add', '--', ...files]);
|
|
60
|
+
if (!added.ok) {
|
|
61
|
+
run(root, 'git', ['checkout', base]);
|
|
62
|
+
return { status: 'refused', next: [`could not stage the files: ${added.out.slice(0, 200)}`] };
|
|
63
|
+
}
|
|
64
|
+
// Only Tewip's own files are committed: anything else the person had in progress stays theirs.
|
|
65
|
+
const committed = run(root, 'git', ['commit', '-m', input.title, '-m', requestBody(input.body, files, input.issue), '--only', '--', ...files]);
|
|
66
|
+
if (!committed.ok) {
|
|
67
|
+
run(root, 'git', ['checkout', base]);
|
|
68
|
+
return { status: 'refused', next: [`could not commit: ${committed.out.slice(0, 200)}`] };
|
|
69
|
+
}
|
|
70
|
+
const next = [];
|
|
71
|
+
const pushed = run(root, 'git', ['push', '-u', 'origin', branch]);
|
|
72
|
+
if (!pushed.ok) {
|
|
73
|
+
return {
|
|
74
|
+
status: 'committed',
|
|
75
|
+
branch,
|
|
76
|
+
next: [`the work is committed on ${branch}, but pushing failed: ${pushed.out.slice(0, 200)}`, `push it yourself with: git push -u origin ${branch}`],
|
|
77
|
+
};
|
|
78
|
+
}
|
|
79
|
+
const gh = run(root, 'gh', ['pr', 'create', '--draft', '--base', base, '--head', branch, '--title', input.title, '--body', requestBody(input.body, files, input.issue)]);
|
|
80
|
+
if (gh.ok) {
|
|
81
|
+
const url = (gh.out.match(/https:\/\/\S+/) ?? [])[0];
|
|
82
|
+
return { status: 'opened', branch, url, next: [`review it: ${url ?? branch}`, `back to where you were: git checkout ${base}`] };
|
|
83
|
+
}
|
|
84
|
+
next.push(`${branch} is pushed, but the pull request was not opened: ${gh.out.slice(0, 200)}`);
|
|
85
|
+
next.push(`open it with: gh pr create --draft --base ${base} --head ${branch}`);
|
|
86
|
+
next.push(`back to where you were: git checkout ${base}`);
|
|
87
|
+
return { status: 'committed', branch, next };
|
|
88
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tewip-ai",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.13.0",
|
|
4
4
|
"description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"test-failures",
|