tewip-ai 0.10.0 → 0.12.0

package/README.md

@@ -166,6 +166,57 @@ that lost its markup) and **5 of 5 harmless changes ignored** (capitals, extra s
 implemented as a link, reordered sections). No designer has judged the output yet, so the roadmap's
 own metric is still open.
 
+## More runners: Robot Framework, JMeter, k6 (new)
+
+Tewip now reads **Robot Framework's output.xml**, **JMeter's .jtl** and a **k6 summary** as well as
+JUnit XML, TRX, NUnit and Playwright's own reports. A load test does not fail test by test, so a
+JMeter report becomes one line per request ("4 of 5 requests failed, an assertion failed") and a k6
+run becomes one record per crossed threshold, with the number that crossed it.
+
+Appium and Selenium need no importer: those suites report through Mocha, pytest or JUnit, which
+Tewip already reads, and their own error messages ("Could not find a connected device", "stale
+element reference") are already understood.
+
+Each reader was checked against real reports published in public repositories, which is how three
+defects were found before release - a Robot timestamp that is not a date, k6's numbers living under
+`values`, and a JMeter line that read "mostly 200" when an assertion had failed.
+
+## Build a feature, test first (new)
+
+```bash
+tewip build "naturalsize should take a sep argument that goes between the number and the unit"
+tewip build "..." --dry-run     # prove it, show the patch, leave nothing on disk
+```
+
+Tewip writes the **test first** and runs it. **If that test passes, it stops and keeps nothing** —
+a test that passes before the feature exists is testing nothing. Then it writes the smallest code
+that makes the test pass, and keeps the work only if the new test passes *and* nothing that was
+passing has started to fail. Otherwise every file goes back byte for byte, including the test it
+wrote. Nothing is ever committed.
+
+Tried on two real public projects: it added a trailing-slash rule to **unjs/ufo** (4 lines,
+TypeScript, $0.18) and a `sep` argument to **python-humanize** (2 lines, Python, $0.10), each with
+its own failing-then-passing test and the project's whole suite still green. Asked for something
+that already worked, it stopped after one model call and kept nothing.
+
+The old promise was *"it can't touch your code"*. The new one is *"it can't change your code without
+showing you it works"*.
+
+## Before anyone reads the diff (new)
+
+```bash
+tewip review --diff main...HEAD
+```
+
+Which tests cover this change and **how** (they import the file, they import the package that
+exports it, or their words merely match); which of them have flaked before, so a red run is not
+blamed on your change; what has gone wrong in these areas before; what nothing tests at all; and the
+command to run the telling tests first. From your project's own history, so it costs nothing.
+
+Measured on **16 real commits** from two public projects, where the developers changed source *and*
+its test: given only the source files, Tewip **named the right test in 16 of 16, ranked it first in
+12**.
+
 ## Which flaky tests to quarantine, and what they cost (new)
 
 ```bash

package/TEAM-GUIDE.md

@@ -127,6 +127,8 @@ collects environment failures in one issue, and (optionally) opens a fix pull re
 | `tewip spec "<feature>"` | An epic, its user stories and acceptance criteria, grounded in your code; files them in Jira with `--project PROJ --create` | Before the work is broken down |
 | `tewip report report.xml --gate` | The CI command for GitLab, Jenkins, CircleCI, Azure, Bitbucket and the rest: diagnoses the run, comments on the GitLab merge request, and fails the build only on failures that need a person | Your CI is not GitHub Actions |
 | `tewip watch --cmd "npm test"` | Runs your tests when you save a file and explains each failure straight away | While you are writing code |
+| `tewip build "<what it should do>"` | Writes a feature test first, then the code, and keeps it only if the test failed without it and passes with it | You want a small feature built and proved |
+| `tewip review --diff main...HEAD` | Which tests cover your change, which have flaked before, what nothing tests, and what to run first | Before you open a pull request |
 | `tewip flaky --days 30` | Which tests to quarantine, which to fix, what they cost in CI minutes - and when a quarantined one can come back | Monday morning |
 | `tewip note --days 7` | The weekly quality note for a lead, from runs already diagnosed (no model calls, free) | Monday morning |
 | `tewip fix --code report.xml` | Opt-in: proposes the smallest change to application code for a real bug, and keeps it only if the failing test passes and the whole suite still passes | "It is a real bug and I want a starting patch" |
@@ -140,7 +142,7 @@ Notes that save you time:
 - `write` never touches your application code. It only adds new test files, and you review and
   commit them yourself.
 - Everything it writes locally goes to the `.tewip` folder in your project.
-- Work items go to **Jira, Rally or GitHub Issues** - set `TEWIP_TRACKER`, or just the credentials
+- Work items go to **Jira, Rally, GitHub Issues, Azure Boards or Linear** - set `TEWIP_TRACKER`, or just the credentials
   of the one you use. GitHub Issues uses your existing `gh` login, so it needs no new secret.
 - Jira works with **Cloud, Server and Data Center**. To read tickets from a tracker anyone can read
   (many open-source ones), `TEWIP_JIRA_URL` alone is enough - no account. Writing needs a token.

package/dist/cli/tewip.js

@@ -67,6 +67,17 @@ Usage:
                                     the GitLab merge request, tells Slack or Teams only when a
                                     failure needs a person, and with --gate fails the build only on
                                     those failures.
+  tewip build "<what it should do>" [--dry-run] [--repairs 2] [--out feature.patch]
+                                    Writes a feature, test first: the test must fail without the
+                                    code and pass with it, and every test that was passing must
+                                    still pass - otherwise everything goes back byte for byte.
+                                    Nothing is committed.
+  tewip review [--diff main...HEAD] [--out review.md]
+                                    Before anyone reads the diff: which tests cover this change,
+                                    which of them have flaked before, what has gone wrong in these
+                                    areas, what nothing tests at all, and the command that runs the
+                                    telling tests first. From this project's own history, so it
+                                    costs nothing.
   tewip watch [--cmd "npm test"] [--report <path>] [--paths src,tests]
                                     Beside your editor: runs your tests when a file changes (or
                                     watches for new reports) and explains each failure as it
@@ -92,7 +103,9 @@ Usage:
   tewip --version                   Print the version.
 
 Reports it reads: Playwright, pytest/unittest, Jest, Vitest, Mocha, node:test, JUnit/TestNG
-(Maven, Gradle), Go (gotestsum), .NET (TRX or JUnit), RSpec, PHPUnit, Rust (nextest).
+(Maven, Gradle), Go (gotestsum), .NET (TRX or JUnit), NUnit 3, RSpec, PHPUnit, Rust (nextest),
+Robot Framework (output.xml), JMeter (.jtl) and k6 (--summary-export). Appium, Selenium and
+WebdriverIO suites report through one of the above, and their own error messages are understood.
 Set OPENAI_API_KEY to let the AI layer decide what the rules leave open.`;
 const flag = (args, name) => {
     const i = args.indexOf(`--${name}`);
@@ -248,6 +261,36 @@ async function main() {
                 process.exitCode = 1;
             return;
         }
+        case 'build': {
+            const feature = args.filter((a, i) => !a.startsWith('--') && !['--repairs', '--out', '--max-lines'].includes(args[i - 1] ?? '')).join(' ').trim();
+            if (!feature)
+                throw new Error('usage: tewip build "<what the feature should do>" [--dry-run] [--repairs 2] [--out feature.patch]');
+            const { runBuild, renderBuild } = await import('../engine/build/build-command.js');
+            const dryRun = args.includes('--dry-run');
+            const result = await runBuild({
+                feature,
+                root: process.cwd(),
+                dryRun,
+                repairs: flag(args, 'repairs') === undefined ? 2 : Math.max(0, Number(flag(args, 'repairs')) || 0),
+                maxLines: flag(args, 'max-lines') ? Number(flag(args, 'max-lines')) : undefined,
+                out: flag(args, 'out'),
+            });
+            console.log(`
+${renderBuild(result, feature, dryRun)}`);
+            if (result.status !== 'built')
+                process.exitCode = 1;
+            return;
+        }
+        case 'review': {
+            const { runReview } = await import('../engine/review/review-command.js');
+            await runReview({
+                root: process.cwd(),
+                diff: flag(args, 'diff') ?? 'main...HEAD',
+                out: flag(args, 'out'),
+                name: flag(args, 'name'),
+            });
+            return;
+        }
         case 'watch': {
             const { runWatch } = await import('../engine/watch/watch-command.js');
             await runWatch({

package/dist/engine/autofix/fix-selector.js

@@ -0,0 +1,93 @@
+// ACT + VERIFY for a drifted selector in a Cypress, Selenium or WebdriverIO test.
+//
+// The same loop as every other fix Tewip makes: change one thing in the test file, run that test,
+// keep the change only if it passes, and otherwise put the file back exactly as it was. What is new
+// is where the replacement comes from - the repository, not a page snapshot - which is why these
+// tools can be fixed at all without a trace.
+//
+// It will not touch application code, will not change what the test asserts, and will not offer a
+// replacement it cannot point at in the source or in a commit.
+import { relative, resolve, sep } from 'node:path';
+import { driverLocator } from '../driver-errors.js';
+import { likeItInSource, renamedInGit, tokenOf, writeSelector } from './selector-source.js';
+/**
+ * Where a selector is written in a test: the whole line, so the replacement reads in context.
+ * Cypress and Selenium put the selector in a plain string, so this is simpler than Playwright's
+ * chained calls - and looking for the quoted form keeps it from matching a comment about it.
+ */
+export function findSelectorSites(source, selector) {
+    const quoted = [`'${selector}'`, `"${selector}"`, `\`${selector}\``];
+    const sites = [];
+    source.split('\n').forEach((text, index) => {
+        if (quoted.some((q) => text.includes(q)))
+            sites.push({ line: index + 1, text });
+    });
+    return sites;
+}
+/**
+ * One drifted selector, fixed from what the repository knows. `keep` false checks a suggestion
+ * without leaving it in the file, exactly as the locator fix does.
+ */
+export async function attemptSelectorFix({ record, diagnosis, action, specFile }, deps, { keep = true } = {}) {
+    const title = record.testId.split(' > ').at(-1) ?? record.testId;
+    const result = { testId: record.testId, specFile, status: 'not-eligible', applied: false, log: [] };
+    const stop = (status, why) => ({ ...result, status, log: [...result.log, why] });
+    if (diagnosis.category !== 'LOCATOR_DRIFT' || action.kind !== 'fix-locator') {
+        return stop('not-eligible', `only a renamed element is fixed this way; this one is ${diagnosis.category} (${action.kind})`);
+    }
+    const rel = relative(resolve(deps.allowedRoot), resolve(specFile));
+    if (rel.startsWith('..') || rel.startsWith(sep) || /^[a-zA-Z]:/.test(rel))
+        return stop('outside-sandbox', `refused: ${specFile} is outside ${deps.allowedRoot}`);
+    // What the tool said it was looking for.
+    const selector = driverLocator(record.errorMessage) ?? '';
+    if (!selector)
+        return stop('no-selector', 'the error does not say which selector was not found');
+    result.log.push(`the test looked for ${selector}`);
+    // What the line in the test really holds: Selenium prints "css selector=#x" but writes "#x".
+    const written = /^[a-z ]+=\s*/i.test(selector) ? selector.replace(/^[a-z ]+=\s*/i, '') : selector;
+    const token = tokenOf(selector);
+    if (!token) {
+        return stop('not-identifiable', `${selector} is not an id, class or data attribute, so there is nothing in the source to compare it against. A person should look at the page.`);
+    }
+    // Git first: somebody renamed it, and the commit says to what. The source is the weaker answer.
+    const found = renamedInGit(deps.projectRoot, token) ?? likeItInSource(deps.projectRoot, token, deps.sources);
+    if (!found) {
+        return stop('no-replacement', `nothing in the source or in the history looks like "${token.value}" renamed. Tewip will not invent a selector.`);
+    }
+    const evidence = found.how === 'renamed in git'
+        ? `commit ${found.commit} changed "${token.value}" to "${found.value}" in ${found.file}`
+        : `"${token.value}" is nowhere in the source any more, and ${found.file} now has "${found.value}"`;
+    result.evidence = evidence;
+    result.log.push(evidence);
+    const replacement = writeSelector(token, found.value);
+    const source = deps.readFile(specFile);
+    const sites = findSelectorSites(source, written);
+    const site = sites[0];
+    if (!site)
+        return stop('selector-not-in-test', `${selector} is not written in ${rel}: it comes from a helper or a page object, which a person should change`);
+    result.line = site.line;
+    result.original = site.text;
+    result.replacement = site.text.replace(written, replacement);
+    deps.writeFile(specFile, source.replace(site.text, result.replacement));
+    try {
+        const run = await deps.verify(specFile, title);
+        if (!run.passed) {
+            deps.writeFile(specFile, source);
+            return stop('still-failing', `with ${replacement} the test still failed, so the file was put back: ${run.detail.slice(-200)}`);
+        }
+        result.log.push(`with ${replacement} the test passes`);
+        if (!keep)
+            deps.writeFile(specFile, source);
+        return { ...result, status: 'fixed', applied: keep };
+    }
+    catch (error) {
+        deps.writeFile(specFile, source);
+        throw error;
+    }
+}
+/** The one-line diff a reviewer reads. */
+export function selectorFixDiff(result) {
+    if (!result.original || !result.replacement)
+        return '';
+    return [`--- a/${result.specFile}`, `+++ b/${result.specFile}`, `@@ line ${result.line} @@`, `-${result.original.trim()}`, `+${result.replacement.trim()}`].join('\n');
+}

package/dist/engine/autofix/selector-source.js

@@ -0,0 +1,162 @@
+// Phase 5, for the tools that do not have a page snapshot: Cypress, Selenium and WebdriverIO.
+//
+// Playwright's traces tell Tewip what was on the page, so a drifted locator can be matched against
+// the real elements. A Cypress or Selenium failure in a JUnit report carries none of that - only
+// "Expected to find element: '#submit-btn', but never found it". Inventing a replacement from that
+// would be exactly the kind of guess this project exists not to make.
+//
+// So the answer comes from the repository instead. If a test looks for `#submit-btn` and the app
+// now renders `id="submit-button"`, the new name is in the application source, and git knows which
+// commit renamed it. That is evidence, not a guess - and the fix is still only kept if the test
+// passes afterwards.
+import { execFileSync } from 'node:child_process';
+import { readFileSync } from 'node:fs';
+/** What a CSS or XPath selector is really identifying, when it is something identifiable. */
+export function tokenOf(selector) {
+    // Selenium and WebdriverIO name the strategy first: "css selector=#submit-btn", "id=submit-btn".
+    const strategy = /^(css selector|css|xpath|id|name|class name|link text|partial link text|tag name)\s*=\s*(.+)$/i.exec(selector.trim());
+    const s = (strategy ? strategy[2] : selector).trim();
+    if (strategy) {
+        const method = strategy[1].toLowerCase();
+        if (method === 'id')
+            return { kind: 'id', value: s };
+        if (method === 'class name')
+            return { kind: 'class', value: s.split(/\s+/)[0] };
+        if (method === 'link text' || method === 'partial link text')
+            return { kind: 'text', value: s };
+    }
+    const data = /\[\s*([\w-]*data[\w-]*)\s*[~|^$*]?=\s*["']?([^"'\]]+)["']?\s*\]/i.exec(s);
+    if (data)
+        return { kind: 'data', attribute: data[1], value: data[2] };
+    const id = /^#([\w-]+)$|\[\s*id\s*=\s*["']([^"']+)["']\s*\]|@id\s*=\s*["']([^"']+)["']/i.exec(s);
+    if (id)
+        return { kind: 'id', value: id[1] ?? id[2] ?? id[3] };
+    const cls = /^\.([\w-]+)$|\[\s*class\s*=\s*["']([^"']+)["']\s*\]/i.exec(s);
+    if (cls)
+        return { kind: 'class', value: (cls[1] ?? cls[2]).split(/\s+/)[0] };
+    const text = /text\(\)\s*=\s*["']([^"']+)["']|contains\(\s*(?:text\(\)|\.)\s*,\s*["']([^"']+)["']/i.exec(s);
+    if (text)
+        return { kind: 'text', value: text[1] ?? text[2] };
+    return null;
+}
+/** How close two names are, 0 to 1, by the words in them - "submit-btn" and "submit-button". */
+export function closeness(a, b) {
+    const words = (s) => new Set(s.toLowerCase().split(/[^a-z0-9]+/).filter(Boolean));
+    const left = words(a);
+    const right = words(b);
+    if (left.size === 0 || right.size === 0)
+        return 0;
+    let shared = 0;
+    for (const w of left)
+        if (right.has(w))
+            shared++;
+    // A shared prefix counts too: "submitBtn" and "submitButton" have no word in common.
+    const prefix = commonPrefix(a.toLowerCase(), b.toLowerCase()) / Math.max(a.length, b.length);
+    return Math.max(shared / Math.max(left.size, right.size), prefix);
+}
+function commonPrefix(a, b) {
+    let i = 0;
+    while (i < a.length && i < b.length && a[i] === b[i])
+        i++;
+    return i;
+}
+const git = (root, args) => {
+    try {
+        return execFileSync('git', args, { cwd: root, encoding: 'utf8', maxBuffer: 32 * 1024 * 1024, timeout: 60_000 });
+    }
+    catch {
+        return '';
+    }
+};
+/**
+ * The commit that removed this value, and what took its place on the same line. This is the
+ * strongest evidence there is: somebody renamed it, and the diff says to what.
+ */
+export function renamedInGit(root, token, maxCommits = 5) {
+    const shas = git(root, ['log', '-S', token.value, '--format=%H', `-n${maxCommits}`, '--', '.'])
+        .split('\n')
+        .map((s) => s.trim())
+        .filter(Boolean);
+    for (const sha of shas) {
+        const diff = git(root, ['show', '--unified=0', '--format=', sha]);
+        if (!diff)
+            continue;
+        let file = '';
+        const removed = [];
+        const added = [];
+        for (const line of diff.split('\n')) {
+            if (line.startsWith('+++ b/'))
+                file = line.slice(6);
+            else if (line.startsWith('-') && line.includes(token.value))
+                removed.push(line);
+            else if (line.startsWith('+'))
+                added.push(line);
+        }
+        if (removed.length === 0 || added.length === 0)
+            continue;
+        // The best replacement is the attribute value on an added line that most resembles the old one.
+        let best = null;
+        for (const line of added) {
+            for (const value of attributeValues(line, token)) {
+                const score = closeness(token.value, value);
+                if (value !== token.value && score >= 0.5 && (!best || score > best.score)) {
+                    best = { value, file, how: 'renamed in git', commit: sha.slice(0, 8), score };
+                }
+            }
+        }
+        if (best)
+            return best;
+    }
+    return null;
+}
+/** Values of the same kind of attribute written on a line of source. */
+function attributeValues(line, token) {
+    const out = [];
+    const attribute = token.kind === 'data' ? token.attribute ?? 'data-[\\w-]+' : token.kind === 'id' ? 'id' : token.kind === 'class' ? 'class(?:Name)?' : '';
+    if (!attribute)
+        return out;
+    const pattern = new RegExp(`${attribute}\\s*=\\s*["'\`{]?\\s*["']?([\\w\\s-]+)["']?`, 'gi');
+    for (const m of line.matchAll(pattern)) {
+        for (const value of m[1].trim().split(/\s+/))
+            if (value)
+                out.push(value);
+    }
+    return out;
+}
+/**
+ * The closest value of the same kind anywhere in the application source now. Weaker than git,
+ * because nobody wrote down that it is a rename - so it is offered only when it is clearly close.
+ */
+export function likeItInSource(root, token, sources, min = 0.6) {
+    let best = null;
+    for (const file of sources) {
+        let text;
+        try {
+            text = readFileSync(file, 'utf8');
+        }
+        catch {
+            continue;
+        }
+        if (text.includes(token.value))
+            return null; // it is still there; this is not a rename
+        for (const line of text.split('\n')) {
+            for (const value of attributeValues(line, token)) {
+                const score = closeness(token.value, value);
+                if (value !== token.value && score >= min && (!best || score > best.score)) {
+                    best = { value, file, how: 'found in the source', score };
+                }
+            }
+        }
+    }
+    return best;
+}
+/** The selector written the way each tool writes it, so the fix reads like the rest of the test. */
+export function writeSelector(token, value) {
+    if (token.kind === 'id')
+        return `#${value}`;
+    if (token.kind === 'class')
+        return `.${value}`;
+    if (token.kind === 'data')
+        return `[${token.attribute}="${value}"]`;
+    return value;
+}

package/dist/engine/build/apply.js

@@ -0,0 +1,99 @@
+// Applying a set of edits to a working tree, and being able to put everything back exactly as it
+// was. Every step of building a feature is provisional until the tests say otherwise, so nothing
+// here writes anything it cannot undo.
+import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { dirname, relative, resolve, sep } from 'node:path';
+import { mayChange } from '../autofix/code-target.js';
+import { changedLines } from '../autofix/code-fix.js';
+export class Workspace {
+    root;
+    touched = new Map();
+    constructor(root) {
+        this.root = resolve(root);
+    }
+    /** Remembers a file's contents the first time it is touched, so it can always be restored. */
+    remember(path) {
+        if (this.touched.has(path))
+            return;
+        this.touched.set(path, existsSync(path) ? readFileSync(path, 'utf8') : null);
+    }
+    write(path, content) {
+        this.remember(path);
+        mkdirSync(dirname(path), { recursive: true });
+        writeFileSync(path, content, 'utf8');
+    }
+    /** Everything back as it was: files restored, files Tewip created removed. */
+    restore() {
+        for (const [path, was] of this.touched) {
+            if (was === null)
+                rmSync(path, { force: true });
+            else
+                writeFileSync(path, was, 'utf8');
+        }
+        this.touched.clear();
+    }
+    /** Keeps the changes, and forgets how to undo them. */
+    keep() {
+        this.touched.clear();
+    }
+    /** What has been touched so far, for the report. */
+    files() {
+        return [...this.touched.keys()].map((p) => relative(this.root, p).split(sep).join('/'));
+    }
+}
+const CR = String.fromCharCode(13);
+const NL = String.fromCharCode(10);
+/**
+ * Applies a model's edits, refusing anything that is not allowed or does not match. Line endings
+ * are compared in one form, because a Windows checkout and a model's answer never agree on them.
+ */
+export function applyEdits(workspace, root, edits, maxLines) {
+    const applied = [];
+    const refused = [];
+    for (const edit of edits) {
+        const path = resolve(root, edit.file);
+        const rel = relative(resolve(root), path).split(sep).join('/');
+        if (rel.startsWith('..') || resolve(path).startsWith('..')) {
+            refused.push(`${edit.file} is outside the project`);
+            continue;
+        }
+        const allowed = mayChange(path);
+        if (!allowed.allowed) {
+            refused.push(`${rel} may not be changed: ${allowed.why}`);
+            continue;
+        }
+        const exists = existsSync(path);
+        if (!exists) {
+            if (edit.before.trim()) {
+                refused.push(`${rel} does not exist, but the change expects text to already be in it`);
+                continue;
+            }
+            workspace.write(path, edit.after);
+            applied.push({ ...edit, path, was: null, changed: edit.after.split(NL).length });
+            continue;
+        }
+        const raw = readFileSync(path, 'utf8');
+        const crlf = raw.includes(CR + NL);
+        const original = crlf ? raw.split(CR + NL).join(NL) : raw;
+        const before = edit.before.split(CR + NL).join(NL);
+        const after = edit.after.split(CR + NL).join(NL);
+        if (!before.trim()) {
+            refused.push(`${rel} already exists, so a change to it must say which text to replace`);
+            continue;
+        }
+        const occurrences = original.split(before).length - 1;
+        if (occurrences !== 1) {
+            refused.push(`the text to replace in ${rel} ${occurrences === 0 ? 'is not in the file' : `appears ${occurrences} times`}`);
+            continue;
+        }
+        const changed = original.replace(before, after);
+        const touched = changedLines(original, changed);
+        if (touched > maxLines) {
+            refused.push(`the change to ${rel} touches ${touched} lines, which is a rewrite rather than a feature`);
+            continue;
+        }
+        workspace.write(path, crlf ? changed.split(NL).join(CR + NL) : changed);
+        applied.push({ ...edit, path, was: raw, changed: touched });
+    }
+    return { applied, refused };
+}