tewip-ai 0.8.1 → 0.10.0

package/README.md

@@ -166,6 +166,24 @@ that lost its markup) and **5 of 5 harmless changes ignored** (capitals, extra s
 implemented as a link, reordered sections). No designer has judged the output yet, so the roadmap's
 own metric is still open.
 
+## Which flaky tests to quarantine, and what they cost (new)
+
+```bash
+tewip flaky --days 30        # the dossier: quarantine, fix, or watch - with the evidence
+tewip flaky --apply          # writes the quarantine line into the test file
+```
+
+Other tools count retries. Tewip already decided *why* each failure happened, so it can say: this
+test failed in 161 of 647 runs **across 109 different branches** - that is the test, not your
+change - and quarantining it gives the team back 148 CI minutes a week. It refuses to hide a test
+that mostly fails (that one is broken, not flaky) and **never** quarantines a test whose failures
+looked like a real bug.
+
+Measured on three real projects' CI history (Gutenberg twice, Supabase): 76 tests proposed for
+quarantine, **zero of them containing a single real-bug failure**, giving back roughly 34, 148 and
+1,178 CI minutes a week. It also tells you when a quarantined test has stopped failing and should
+come back - the half nobody ever does.
+
 ## Bugs filed back, and test cases where your team keeps them (Phase 10, completed)
 
 ```bash

package/TEAM-GUIDE.md

@@ -127,6 +127,7 @@ collects environment failures in one issue, and (optionally) opens a fix pull re
 | `tewip spec "<feature>"` | An epic, its user stories and acceptance criteria, grounded in your code; files them in Jira with `--project PROJ --create` | Before the work is broken down |
 | `tewip report report.xml --gate` | The CI command for GitLab, Jenkins, CircleCI, Azure, Bitbucket and the rest: diagnoses the run, comments on the GitLab merge request, and fails the build only on failures that need a person | Your CI is not GitHub Actions |
 | `tewip watch --cmd "npm test"` | Runs your tests when you save a file and explains each failure straight away | While you are writing code |
+| `tewip flaky --days 30` | Which tests to quarantine, which to fix, what they cost in CI minutes - and when a quarantined one can come back | Monday morning |
 | `tewip note --days 7` | The weekly quality note for a lead, from runs already diagnosed (no model calls, free) | Monday morning |
 | `tewip fix --code report.xml` | Opt-in: proposes the smallest change to application code for a real bug, and keeps it only if the failing test passes and the whole suite still passes | "It is a real bug and I want a starting patch" |
 | `tewip design <figma link> --url <page>` | Compares a Figma frame against the running page: the words the design specifies and the controls its layer names imply | "Did we build what was designed?" |
@@ -139,6 +140,8 @@ Notes that save you time:
 - `write` never touches your application code. It only adds new test files, and you review and
   commit them yourself.
 - Everything it writes locally goes to the `.tewip` folder in your project.
+- Work items go to **Jira, Rally or GitHub Issues** - set `TEWIP_TRACKER`, or just the credentials
+  of the one you use. GitHub Issues uses your existing `gh` login, so it needs no new secret.
 - Jira works with **Cloud, Server and Data Center**. To read tickets from a tracker anyone can read
   (many open-source ones), `TEWIP_JIRA_URL` alone is enough - no account. Writing needs a token.
 - `tewip jira` shows what it *would* file and creates nothing until you add `--create`. It never

package/dist/cli/tewip.js

@@ -71,6 +71,11 @@ Usage:
                                     Beside your editor: runs your tests when a file changes (or
                                     watches for new reports) and explains each failure as it
                                     happens, without CI.
+  tewip flaky [--days 30] [--apply] [--name <project>] [--out flaky.md]
+                                    Which tests are wasting the team's time, how much CI time they
+                                    cost, and what to do with each: quarantine, fix, or watch. Read
+                                    from runs already diagnosed, so it costs nothing. --apply puts
+                                    the quarantine line in the test file with the evidence above it.
   tewip note [--days 7] [--name <project>] [--out note.md] [--post]
                                     The weekly quality note for a lead, written from the runs Tewip
                                     already diagnosed. No model calls, so it costs nothing.
@@ -254,6 +259,17 @@ async function main() {
             });
             return;
         }
+        case 'flaky': {
+            const { runFlaky } = await import('../engine/flaky/flaky-command.js');
+            await runFlaky({
+                root: process.cwd(),
+                name: flag(args, 'name'),
+                days: Math.max(1, Number(flag(args, 'days')) || 30),
+                apply: args.includes('--apply'),
+                out: flag(args, 'out'),
+            });
+            return;
+        }
         case 'note': {
             const { runNote } = await import('../engine/notify/note-command.js');
             await runNote({

package/dist/engine/flaky/flaky-command.js

@@ -0,0 +1,142 @@
+// `tewip flaky`: the Monday morning answer. Which tests are wasting the team's time, how much, and
+// what to do about each one - read from the runs Tewip already diagnosed, so it costs nothing and
+// cannot re-judge an old failure with today's rules.
+import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { join, resolve } from 'node:path';
+import { datasetFiles, readRunSummaries, readTriageRecords } from '../../reporters/store.js';
+import { flakyTests, readyToRelease, weeklyCost } from './flaky.js';
+import { applyQuarantine, frameworkOf, quarantineFor } from './quarantine.js';
+const DAY_MS = 24 * 60 * 60 * 1000;
+const minutes = (ms) => `${(ms / 60_000).toFixed(1)} CI min`;
+function projects(root, only) {
+    const dir = join(root, '.tewip', 'history');
+    if (only)
+        return [only];
+    return existsSync(dir) ? readdirSync(dir).filter((name) => existsSync(join(dir, name, 'triage.jsonl'))) : [];
+}
+/** The test file on disk, when the report's path still exists in this project. */
+function testFile(root, test) {
+    const direct = resolve(root, test.file);
+    return test.file && existsSync(direct) ? direct : null;
+}
+export async function runFlaky(cmd) {
+    const names = projects(cmd.root, cmd.name);
+    if (names.length === 0) {
+        throw new Error(`no history in ${join(cmd.root, '.tewip', 'history')}. Run \`tewip explain\` or \`tewip report\` on a few runs first: this is read from what those kept.`);
+    }
+    const since = new Date(Date.now() - cmd.days * DAY_MS).toISOString();
+    const all = [];
+    const lines = ['# Flaky tests', '', `The last ${cmd.days} days, from runs Tewip already diagnosed. No tests were run and no model was called.`, ''];
+    for (const name of names) {
+        const history = datasetFiles(join(cmd.root, '.tewip', 'history', name));
+        const records = readTriageRecords(history.triage);
+        const runs = readRunSummaries(history.runs);
+        const tests = flakyTests(records, runs, { since });
+        if (tests.length === 0)
+            continue;
+        all.push(...tests);
+        const cost = weeklyCost(tests, cmd.days);
+        lines.push(`## ${name}`, '', `${tests.length} test(s) failed at least once. About ${cost.minutes} CI minute(s) and ${cost.attempts} failed attempt(s) a week.`, '');
+        const group = (verdict, title, explain) => {
+            const chosen = tests.filter((t) => t.verdict === verdict);
+            if (chosen.length === 0)
+                return;
+            lines.push(`### ${title} (${chosen.length})`, '', explain, '');
+            for (const test of chosen.slice(0, 15)) {
+                lines.push(`- **${test.title}**`);
+                lines.push(`  - ${test.failedRuns} of ${test.totalRuns} run(s), ${test.failedAttempts} failed attempt(s), ${minutes(test.wastedMs)} lost${test.recoveredOnRetry > 0 ? `, passed on a retry ${test.recoveredOnRetry} time(s)` : ''}`);
+                lines.push(`  - Tewip read them as: ${test.categories.map((c) => `${c.category} ×${c.count}`).join(', ')}`);
+                lines.push(`  - Why: ${test.why}`);
+                if (test.file)
+                    lines.push(`  - Where: ${test.file}`);
+            }
+            lines.push('');
+        };
+        group('quarantine', 'Quarantine these', 'Unreliable rather than wrong. Stop them blocking everyone, then fix them on purpose.');
+        group('fix-now', 'Fix these, do not hide them', 'Hiding one of these would hide something real.');
+        group('watch', 'Watch', 'Not enough evidence yet to act.');
+        // Coming back out is the half nobody ever does.
+        const quarantined = readQuarantined(cmd.root);
+        const back = readyToRelease(quarantined, records, runs, since);
+        if (back.length > 0) {
+            lines.push('### Ready to come out of quarantine', '', 'These ran in this period and did not fail once.', '');
+            for (const testId of back)
+                lines.push(`- ${testId.split(' > ').at(-1)} — take its quarantine line out and let it block again.`);
+            lines.push('');
+        }
+    }
+    if (all.length === 0) {
+        console.log(`Nothing failed in the last ${cmd.days} days. Try --days 30.`);
+        return [];
+    }
+    const toQuarantine = all.filter((t) => t.verdict === 'quarantine');
+    if (cmd.apply && toQuarantine.length > 0)
+        lines.push(...applyAll(cmd.root, toQuarantine));
+    else if (toQuarantine.length > 0)
+        lines.push(...suggest(cmd.root, toQuarantine));
+    const report = lines.join('\n');
+    console.log(report);
+    if (cmd.out) {
+        writeFileSync(resolve(cmd.out), `${report}\n`, 'utf8');
+        console.log(`\nWritten to ${cmd.out}`);
+    }
+    return all;
+}
+/** Tests already carrying Tewip's quarantine line, found by reading the test files it knows about. */
+function readQuarantined(root) {
+    const file = join(root, '.tewip', 'quarantined.txt');
+    return existsSync(file) ? readFileSync(file, 'utf8').split('\n').map((l) => l.trim()).filter(Boolean) : [];
+}
+function suggest(root, tests) {
+    const lines = ['### The line to add', ''];
+    for (const test of tests.slice(0, 5)) {
+        const path = testFile(root, test);
+        const framework = path ? frameworkOf(path, readFileSync(path, 'utf8')) : null;
+        if (!path || !framework) {
+            lines.push(`- ${test.title}: ${test.file || 'file not found in this project'} - quarantine it the way this project does.`);
+            continue;
+        }
+        lines.push(`- ${test.title} (${test.file}):`, '```', quarantineFor(test, framework).line, '```');
+    }
+    lines.push('', 'Run `tewip flaky --apply` to put these lines in, each with the evidence above it. Only the named tests are touched.', '');
+    return lines;
+}
+/** Writes the quarantine into the test files, and remembers which tests were quarantined. */
+function applyAll(root, tests) {
+    const lines = ['### Applied', ''];
+    const done = [];
+    for (const test of tests) {
+        const path = testFile(root, test);
+        if (!path) {
+            lines.push(`- ${test.title}: not applied, ${test.file || 'no file'} is not in this project`);
+            continue;
+        }
+        const source = readFileSync(path, 'utf8');
+        const framework = frameworkOf(path, source);
+        if (!framework) {
+            lines.push(`- ${test.title}: not applied, the framework of ${test.file} was not recognised`);
+            continue;
+        }
+        const changed = applyQuarantine(source, test, framework);
+        if (!changed) {
+            lines.push(`- ${test.title}: not applied, the test was not found in ${test.file} (was it renamed?)`);
+            continue;
+        }
+        if (changed === source) {
+            lines.push(`- ${test.title}: already quarantined`);
+            done.push(test.testId);
+            continue;
+        }
+        writeFileSync(path, changed, 'utf8');
+        done.push(test.testId);
+        lines.push(`- ${test.title}: quarantined in ${test.file}`);
+    }
+    if (done.length > 0) {
+        const file = join(root, '.tewip', 'quarantined.txt');
+        const before = readQuarantined(root);
+        writeFileSync(file, [...new Set([...before, ...done])].join('\n') + '\n', 'utf8');
+        lines.push('', `Remembered in ${join('.tewip', 'quarantined.txt')}, so Tewip can tell you when each one is safe to bring back.`);
+    }
+    lines.push('', 'Nothing else in those files was changed. Review the diff before committing: a quarantined test is a decision, not a fix.', '');
+    return lines;
+}

package/dist/engine/flaky/flaky.js

@@ -0,0 +1,104 @@
+const titleOf = (testId) => testId.split(' > ').at(-1) ?? testId;
+const fileOf = (testId, record) => record?.file ?? testId.split(' > ').find((p) => /\.(spec|test)\./.test(p)) ?? '';
+/**
+ * A test's story across the kept runs. `records` are the failures Tewip diagnosed; `runs` say how
+ * often each test ran at all, which is what turns a count into a rate.
+ */
+export function flakyTests(records, runs, options = {}) {
+    const { since = '', minRuns = 2, alwaysFailsAbove = 0.8 } = options;
+    const inWindow = runs.filter((r) => (r.startedAt ?? '') >= since);
+    const runIds = new Set(inWindow.map((r) => r.runId));
+    const branchOf = new Map(inWindow.map((r) => [r.runId, r.branch ?? '']));
+    const failures = records.filter((r) => runIds.has(r.runId) || (!r.runId && !since));
+    const ranIn = new Map();
+    for (const run of inWindow)
+        for (const testId of Object.keys(run.attempts ?? {}))
+            ranIn.set(testId, (ranIn.get(testId) ?? 0) + 1);
+    const byTest = new Map();
+    for (const record of failures)
+        byTest.set(record.testId, [...(byTest.get(record.testId) ?? []), record]);
+    const out = [];
+    for (const [testId, all] of byTest) {
+        const runsFailed = new Set(all.map((r) => r.runId));
+        // A retry that passed: the same test, same run, a later attempt that is not in the failures.
+        const attemptsByRun = new Map();
+        for (const r of all)
+            attemptsByRun.set(r.runId, [...(attemptsByRun.get(r.runId) ?? []), r]);
+        let recovered = 0;
+        for (const [runId, attempts] of attemptsByRun) {
+            const ran = inWindow.find((run) => run.runId === runId)?.attempts?.[testId] ?? attempts.length;
+            if (ran > attempts.length)
+                recovered++;
+        }
+        const counts = new Map();
+        for (const r of all)
+            counts.set(r.diagnosis?.category ?? 'UNKNOWN', (counts.get(r.diagnosis?.category ?? 'UNKNOWN') ?? 0) + 1);
+        const categories = [...counts].map(([category, count]) => ({ category, count })).sort((a, b) => b.count - a.count);
+        const times = all.map((r) => r.timestamp).sort();
+        const branches = [...new Set([...runsFailed].map((id) => branchOf.get(id) ?? '').filter(Boolean))];
+        const totalRuns = ranIn.get(testId) ?? runsFailed.size;
+        const test = {
+            testId,
+            title: titleOf(testId),
+            file: fileOf(testId, all[0]),
+            failedRuns: runsFailed.size,
+            totalRuns,
+            failedAttempts: all.length,
+            wastedMs: all.reduce((sum, r) => sum + (r.duration ?? 0), 0),
+            recoveredOnRetry: recovered,
+            branches,
+            categories,
+            firstSeen: times[0] ?? '',
+            lastSeen: times.at(-1) ?? '',
+            verdict: 'watch',
+            why: '',
+        };
+        Object.assign(test, decide(test, { minRuns, alwaysFailsAbove }));
+        out.push(test);
+    }
+    return out.sort((a, b) => b.wastedMs - a.wastedMs || b.failedRuns - a.failedRuns);
+}
+const share = (test) => (test.totalRuns > 0 ? test.failedRuns / test.totalRuns : 1);
+const flakyShare = (test) => {
+    const flaky = test.categories.find((c) => c.category === 'TIMING')?.count ?? 0;
+    return test.failedAttempts > 0 ? flaky / test.failedAttempts : 0;
+};
+/**
+ * The decision, and why. Three rules, all conservative:
+ *   a test that mostly fails is broken, not flaky - hiding it hides a real problem;
+ *   a test whose failures Tewip read as the application being wrong is never quarantined;
+ *   quarantine needs the test to have failed in several runs, and to be unreliable rather than
+ *   consistently wrong.
+ */
+function decide(test, { minRuns, alwaysFailsAbove }) {
+    const rate = share(test);
+    const percent = `${Math.round(rate * 100)}% of the ${test.totalRuns} run(s) it was in`;
+    const realBugs = test.categories.find((c) => c.category === 'REAL_BUG')?.count ?? 0;
+    if (realBugs > 0 && realBugs >= test.failedAttempts / 2) {
+        return { verdict: 'fix-now', why: `${realBugs} of its ${test.failedAttempts} failures looked like the application being wrong, not the test. Quarantining it would hide a bug.` };
+    }
+    if (rate > alwaysFailsAbove && test.totalRuns >= minRuns) {
+        return { verdict: 'fix-now', why: `it fails in ${percent}. That is not flakiness, it is a test that does not work - hiding it would hide whatever it is telling you.` };
+    }
+    if (test.failedRuns >= minRuns && flakyShare(test) >= 0.5) {
+        const spread = test.branches.length > 1 ? ` It failed on ${test.branches.length} different branches, so it is the test rather than one person's change.` : '';
+        return { verdict: 'quarantine', why: `it failed in ${percent}, and Tewip read most of those as the test being unreliable.${spread}` };
+    }
+    return { verdict: 'watch', why: `it failed in ${percent}: not often enough, or not consistently enough one thing, to act on yet.` };
+}
+/** What a week of this costs, so the number means something to whoever pays for CI. */
+export function weeklyCost(tests, days) {
+    const wasted = tests.reduce((sum, t) => sum + t.wastedMs, 0);
+    const attempts = tests.reduce((sum, t) => sum + t.failedAttempts, 0);
+    const weeks = Math.max(days / 7, 0.1);
+    return { minutes: Math.round(wasted / 60_000 / weeks), attempts: Math.round(attempts / weeks) };
+}
+/**
+ * Tests that were quarantined and have stopped failing. Nobody ever takes a test out of quarantine,
+ * so this looks for the ones that have earned it: no failure in the window, and they ran.
+ */
+export function readyToRelease(quarantined, records, runs, since) {
+    const failed = new Set(records.filter((r) => r.timestamp >= since).map((r) => r.testId));
+    const ran = new Set(runs.filter((r) => (r.startedAt ?? '') >= since).flatMap((r) => Object.keys(r.attempts ?? {})));
+    return quarantined.filter((testId) => ran.has(testId) && !failed.has(testId));
+}

package/dist/engine/flaky/quarantine.js

@@ -0,0 +1,77 @@
+/** Which framework a test file belongs to, from its name and its contents. */
+export function frameworkOf(file, source) {
+    if (/\.py$/.test(file))
+        return 'pytest';
+    if (/\.java$|\.kt$/.test(file))
+        return 'junit';
+    if (/\.cs$/.test(file))
+        return 'dotnet';
+    if (/_test\.go$/.test(file))
+        return 'go';
+    if (/@playwright\/test|from ['"]playwright/.test(source))
+        return 'playwright';
+    if (/from ['"]vitest['"]|\bvi\./.test(source))
+        return 'vitest';
+    if (/@jest\/globals|jest\./.test(source))
+        return 'jest';
+    if (/\brequire\(['"]mocha|describe\(/.test(source))
+        return 'mocha';
+    return /\.[cm]?[jt]sx?$/.test(file) ? 'jest' : null;
+}
+const MARK = {
+    playwright: (title) => `test.fixme(true, ${JSON.stringify(`quarantined by Tewip: ${title}`)});`,
+    jest: () => 'test.skip',
+    vitest: () => 'test.skip',
+    mocha: () => 'it.skip',
+    pytest: (title) => `@pytest.mark.skip(reason=${JSON.stringify(`quarantined by Tewip: ${title}`)})`,
+    junit: (title) => `@Disabled(${JSON.stringify(`quarantined by Tewip: ${title}`)})`,
+    dotnet: (title) => `[Fact(Skip = ${JSON.stringify(`quarantined by Tewip: ${title}`)})]`,
+    go: () => 't.Skip("quarantined by Tewip")',
+};
+/** What to write, and the note that goes above it. */
+export function quarantineFor(test, framework) {
+    const rate = test.totalRuns > 0 ? `${test.failedRuns} of ${test.totalRuns} runs` : `${test.failedRuns} run(s)`;
+    return {
+        line: MARK[framework](test.title),
+        comment: [
+            `Quarantined by Tewip on ${new Date().toISOString().slice(0, 10)}: failed in ${rate}, ${Math.round(test.wastedMs / 60_000)} CI minute(s) lost.`,
+            test.branches.length > 1 ? `It failed on ${test.branches.length} branches, so it is the test and not one change.` : '',
+            'Take this line out when the cause is fixed. `tewip flaky` will say when it has stopped failing.',
+        ].filter(Boolean),
+    };
+}
+/**
+ * The file with the test quarantined, or null when the test could not be found in it. Only the one
+ * test named is touched; everything else in the file stays byte for byte as it was.
+ */
+export function applyQuarantine(source, test, framework) {
+    const nl = source.includes('\r\n') ? '\r\n' : '\n';
+    const lines = source.split(nl);
+    const quoted = test.title.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+    const declaration = new RegExp(`^(\\s*)((?:async\\s+)?(?:export\\s+)?(?:test|it|def test_|public\\s+void|func\\s+Test)\\b[^\\n]*${quoted})`, 'i');
+    const index = lines.findIndex((line) => declaration.test(line) || line.includes(`"${test.title}"`) || line.includes(`'${test.title}'`));
+    if (index < 0)
+        return null;
+    if (lines.slice(Math.max(0, index - 4), index + 1).some((l) => /quarantined by tewip/i.test(l)))
+        return source; // already done
+    const indent = (lines[index].match(/^\s*/) ?? [''])[0];
+    const { line, comment } = quarantineFor(test, framework);
+    const commentMark = framework === 'pytest' ? '#' : '//';
+    const note = comment.map((c) => `${indent}${commentMark} ${c}`);
+    // Where the mark goes differs: an annotation sits above the test, a skip replaces the call.
+    if (framework === 'jest' || framework === 'vitest' || framework === 'mocha') {
+        const replaced = lines[index].replace(/\b(test|it)(\.\w+)?\(/, `${line}(`);
+        if (replaced === lines[index])
+            return null;
+        return [...lines.slice(0, index), ...note, replaced, ...lines.slice(index + 1)].join(nl);
+    }
+    if (framework === 'playwright') {
+        // Playwright's fixme goes inside the test body, so the test still reports itself as skipped.
+        const body = lines[index].includes('=>') ? index : -1;
+        if (body < 0)
+            return null;
+        return [...lines.slice(0, index), ...note, lines[index], `${indent}  ${line}`, ...lines.slice(index + 1)].join(nl);
+    }
+    // pytest, JUnit, .NET: the annotation goes on its own line above the test.
+    return [...lines.slice(0, index), ...note, `${indent}${line}`, ...lines.slice(index)].join(nl);
+}

package/dist/engine/work/bug-command.js

@@ -15,8 +15,8 @@ import { importBlobReportDir } from '../import/blob.js';
 import { llmFromEnv } from '../llm/client.js';
 import { buildItems } from '../../outlets/items.js';
 import { ciFacts } from '../ci/platform.js';
-import { Jira, jiraFromEnv, jiraHelp } from './jira.js';
 import { TEWIP_LABEL } from './jira-command.js';
+import { trackerFromEnv, trackerHelp } from './trackers.js';
 /** Bugs Tewip filed carry this as well, so a second run finds them and does not file again. */
 export const BUG_LABEL = 'tewip-bug';
 const isBlobDir = (path) => statSync(path).isDirectory() && readdirSync(path).some((f) => /^report.*\.zip$/.test(f));
@@ -59,11 +59,11 @@ export function bugSummary(item) {
     return (same ? item.title : `${item.title} (${where})`).replace(/\s+/g, ' ').slice(0, 250);
 }
 export async function runBugs(cmd) {
-    const config = jiraFromEnv();
-    // A dry run answers the question people ask first - "what would it file?" - and needs no account
-    // at all. Only --create talks to Jira.
-    if (cmd.create && !config?.token)
-        throw new Error(`filing a bug needs a token: showing what would be filed does not.\n${jiraHelp()}`);
+    // Jira, Rally or GitHub Issues - whichever this team uses. A dry run answers the question people
+    // ask first ("what would it file?") and needs no account at all; only --create writes.
+    const chosen = trackerFromEnv(cmd.projectKey);
+    if (cmd.create && !chosen?.tracker.canWrite())
+        throw new Error(`filing a bug needs a tracker that can be written to; showing what would be filed does not.\n${trackerHelp()}`);
     const path = resolve(cmd.report);
     if (!existsSync(path))
         throw new Error(`no such report: ${cmd.report}`);
@@ -87,41 +87,36 @@ export async function runBugs(cmd) {
         console.log(`${items.length} failure(s), and none of them is a real bug Tewip is confident about (${kinds}). Nothing was filed, which is the point.`);
         return [];
     }
-    const jira = config?.token ? new Jira(config) : null;
-    let type = 'Bug';
+    const tracker = chosen?.tracker.canWrite() ? chosen.tracker : null;
     let existing = [];
-    if (jira) {
-        const types = await jira.issueTypes(cmd.projectKey);
-        type = ['Bug', 'Defect', 'Task'].find((t) => types.includes(t)) ?? types[0];
-        if (!type)
-            throw new Error(`no issue type available in ${cmd.projectKey}`);
-        existing = await jira.findByLabel(cmd.projectKey, BUG_LABEL);
+    if (tracker && chosen) {
+        lines.push(`Tracker: ${tracker.name}`, '');
+        existing = await tracker.mine(chosen.project, BUG_LABEL);
     }
     else {
-        lines.push('Jira was not contacted: this is what would be filed. Set TEWIP_JIRA_URL, TEWIP_JIRA_EMAIL and TEWIP_JIRA_TOKEN, then add --create.', '');
+        lines.push('No tracker was contacted: this is what would be filed. Set one up (see `tewip bugs` without --create for the settings), then add --create.', '');
     }
     let filed = 0;
     for (const item of bugs.slice(0, cmd.max)) {
         const summary = bugSummary(item);
-        const already = existing.find((i) => i.summary === summary);
+        const already = existing.find((i) => i.title === summary);
         if (already) {
             lines.push(`  ${item.title}  already filed as ${already.key}`);
-            if (cmd.create && jira) {
-                await jira.comment(already.key, seenAgain(item, facts.runUrl)).catch(() => lines.push(`    (could not add the note to ${already.key})`));
+            if (cmd.create && tracker) {
+                await tracker.comment(already.key, seenAgain(item, facts.runUrl)).catch(() => lines.push(`    (could not add the note to ${already.key})`));
                 lines.push(`    added a note that it happened again`);
             }
             continue;
         }
-        if (!cmd.create || !jira) {
+        if (!cmd.create || !tracker || !chosen) {
             lines.push(`  would file: ${summary}`);
             lines.push(`      because: ${item.diagnosis.category} at ${item.diagnosis.confidence.toFixed(2)} - ${item.errorHead.replace(/\s+/g, ' ').slice(0, 100)}`);
             continue;
         }
-        const key = await jira.createIssue({
-            projectKey: cmd.projectKey,
-            type,
-            summary,
-            description: bugBody(item, facts.runUrl),
+        const key = await tracker.create(chosen.project, {
+            kind: 'bug',
+            title: summary,
+            body: bugBody(item, facts.runUrl),
             labels: [TEWIP_LABEL, BUG_LABEL, `tewip-${item.component.replace(/[^\w-]+/g, '-')}`.slice(0, 40)],
         });
         lines.push(`  filed ${key}: ${summary}`);
@@ -129,7 +124,7 @@ export async function runBugs(cmd) {
     }
     const left = items.length - bugs.length;
     lines.push('', `${bugs.length} of ${items.length} failure(s) look like real bugs; ${left} were flaky tests, environment or test data and were deliberately not filed.`);
-    lines.push(cmd.create ? `${filed} ticket(s) created. Tewip never moves, assigns or closes one.` : 'Nothing was created. Add --create to file them.');
+    lines.push(cmd.create ? `${filed} ticket(s) created in ${tracker?.name}. Tewip never moves, assigns or closes one.` : 'Nothing was created. Add --create to file them.');
     console.log(lines.join('\n'));
     return bugs;
 }

package/dist/engine/work/github-issues.js

@@ -0,0 +1,63 @@
+// GitHub Issues, for the many teams whose backlog is simply their repository.
+//
+// It uses the token in the environment, or the `gh` CLI login the person already has, so a team on
+// GitHub needs no new account and no new secret. The restraint is the same: create, comment on what
+// it created, find what it filed before by its label. It never closes an issue, never assigns
+// anyone, and never touches an issue somebody else opened.
+import { spawnSync } from 'node:child_process';
+export function githubIssuesFromEnv(env = process.env) {
+    const repo = (env.TEWIP_GITHUB_REPO ?? env.GITHUB_REPOSITORY ?? '').trim();
+    if (!repo)
+        return null;
+    return { repo, token: (env.TEWIP_GITHUB_TOKEN ?? env.GITHUB_TOKEN ?? env.GH_TOKEN ?? '').trim() || ghCliToken() };
+}
+/** The token the `gh` CLI already holds, so a person logged in there needs nothing else. */
+function ghCliToken() {
+    const r = spawnSync('gh auth token', { shell: true, encoding: 'utf8', timeout: 20_000 });
+    return r.status === 0 ? (r.stdout ?? '').trim() : '';
+}
+/** The kind of work is a label here: GitHub has one issue type and many labels. */
+const KIND_LABEL = { 'test-case': 'test-case', bug: 'bug', story: 'story', epic: 'epic' };
+export class GitHubIssues {
+    name = 'GitHub Issues';
+    config;
+    constructor(config) {
+        this.config = config;
+    }
+    canWrite() {
+        return Boolean(this.config.repo && this.config.token);
+    }
+    async request(method, path, body) {
+        const res = await fetch(`https://api.github.com${path}`, {
+            method,
+            headers: {
+                Authorization: `Bearer ${this.config.token}`,
+                Accept: 'application/vnd.github+json',
+                'X-GitHub-Api-Version': '2022-11-28',
+                'User-Agent': 'tewip',
+                ...(body ? { 'Content-Type': 'application/json' } : {}),
+            },
+            body: body ? JSON.stringify(body) : undefined,
+        });
+        if (!res.ok)
+            throw new Error(`GitHub ${method} ${path} -> ${res.status}: ${(await res.text()).slice(0, 300)}`);
+        return (res.status === 204 ? undefined : await res.json());
+    }
+    async mine(_project, label) {
+        // Closed ones count too: an issue Tewip filed and a person closed must not be filed again.
+        const issues = await this.request('GET', `/repos/${this.config.repo}/issues?labels=${encodeURIComponent(label)}&state=all&per_page=100`);
+        return issues.map((i) => ({ key: `#${i.number}`, title: i.title, url: i.html_url }));
+    }
+    async create(_project, item) {
+        const created = await this.request('POST', `/repos/${this.config.repo}/issues`, {
+            title: item.title.slice(0, 250),
+            body: item.body.slice(0, 60_000),
+            labels: [...new Set([...(item.labels ?? []), KIND_LABEL[item.kind], ...(item.priority ? [`priority:${item.priority.toLowerCase()}`] : [])])],
+        });
+        return `#${created.number}`;
+    }
+    async comment(key, text) {
+        const number = key.replace('#', '');
+        await this.request('POST', `/repos/${this.config.repo}/issues/${number}/comments`, { body: text.slice(0, 60_000) });
+    }
+}

package/dist/engine/work/rally.js

@@ -0,0 +1,84 @@
+import { priorityWord } from './tracker.js';
+/** Rally's own name for each kind of item, and the endpoint that creates it. */
+const TYPE = {
+    'test-case': 'testcase',
+    bug: 'defect',
+    story: 'hierarchicalrequirement',
+    epic: 'portfolioitem/feature',
+};
+/** The field that holds the words, which differs by type. */
+const BODY_FIELD = { 'test-case': 'Description', bug: 'Description', story: 'Description', epic: 'Description' };
+export function rallyFromEnv(env = process.env) {
+    const apiKey = env.TEWIP_RALLY_KEY?.trim();
+    if (!apiKey)
+        return null;
+    return {
+        baseUrl: (env.TEWIP_RALLY_URL ?? 'https://rally1.rallydev.com').trim().replace(/\/+$/, ''),
+        apiKey,
+        project: (env.TEWIP_RALLY_PROJECT ?? '').trim(),
+    };
+}
+export class Rally {
+    name = 'Rally';
+    config;
+    constructor(config) {
+        this.config = config;
+    }
+    canWrite() {
+        return Boolean(this.config.apiKey && this.config.project);
+    }
+    async request(method, path, body) {
+        const res = await fetch(`${this.config.baseUrl}/slm/webservice/v2.0${path}`, {
+            method,
+            headers: { ZSESSIONID: this.config.apiKey, Accept: 'application/json', ...(body ? { 'Content-Type': 'application/json' } : {}) },
+            body: body ? JSON.stringify(body) : undefined,
+        });
+        if (!res.ok)
+            throw new Error(`Rally ${method} ${path} -> ${res.status}: ${(await res.text()).slice(0, 300)}`);
+        const json = (await res.json());
+        // Rally answers 200 with the errors inside the body, so they have to be read out of it.
+        const errors = json?.CreateResult?.Errors ?? json?.QueryResult?.Errors ?? json?.OperationResult?.Errors ?? [];
+        if (errors.length > 0)
+            throw new Error(`Rally ${method} ${path}: ${errors.join('; ').slice(0, 300)}`);
+        return json;
+    }
+    /** Items Tewip filed before, found by the tag it puts on everything it creates. */
+    async mine(project, label) {
+        const found = [];
+        for (const kind of ['testcase', 'defect', 'hierarchicalrequirement']) {
+            const query = encodeURIComponent(`(Tags.Name = "${label}")`);
+            const scope = project || this.config.project;
+            const path = `/${kind}?query=${query}&fetch=FormattedID,Name&pagesize=200${scope ? `&project=${encodeURIComponent(scope)}` : ''}`;
+            try {
+                const body = await this.request('GET', path);
+                for (const r of body.QueryResult?.Results ?? [])
+                    found.push({ key: String(r.FormattedID ?? r._ref ?? ''), title: String(r.Name ?? ''), url: r._ref });
+            }
+            catch {
+                // A workspace without that type is normal; the others still answer.
+            }
+        }
+        return found;
+    }
+    async create(project, item) {
+        const type = TYPE[item.kind];
+        const fields = {
+            Name: item.title.slice(0, 250),
+            [BODY_FIELD[item.kind]]: item.body.slice(0, 32_000),
+            Project: project || this.config.project,
+        };
+        if (item.priority)
+            fields[item.kind === 'bug' ? 'Priority' : 'Risk'] = priorityWord('rally', item.priority);
+        if (item.parent)
+            fields[item.kind === 'story' ? 'PortfolioItem' : 'WorkProduct'] = item.parent;
+        const key = type.split('/')[0];
+        const created = await this.request('POST', `/${type}/create`, { [capitalise(key)]: fields });
+        const made = created.CreateResult?.Object;
+        return String(made?.FormattedID ?? made?._ref ?? '');
+    }
+    /** A note on an item Tewip created. Rally calls it a ConversationPost. */
+    async comment(key, text) {
+        await this.request('POST', '/conversationpost/create', { ConversationPost: { Artifact: key, Text: text.slice(0, 4000) } });
+    }
+}
+const capitalise = (word) => word.charAt(0).toUpperCase() + word.slice(1);

package/dist/engine/work/tracker.js

@@ -0,0 +1,37 @@
+export const TEWIP_LABEL = 'tewip';
+/** Every tracker's own word for a priority, so a P0 does not arrive as "P0" in a tool that says "High". */
+export const PRIORITY_WORDS = {
+    jira: { P0: 'Highest', P1: 'High', P2: 'Medium' },
+    rally: { P0: 'Resolve Immediately', P1: 'High Attention', P2: 'Normal' },
+    github: { P0: 'priority:high', P1: 'priority:medium', P2: 'priority:low' },
+};
+export function priorityWord(tracker, priority) {
+    return PRIORITY_WORDS[tracker]?.[priority] ?? priority;
+}
+/** Which tracker the settings point at. Jira stays the default, because most teams are there. */
+export function trackerNameFromEnv(env = process.env) {
+    const named = (env.TEWIP_TRACKER ?? '').trim().toLowerCase();
+    if (named === 'rally' || named === 'github' || named === 'jira')
+        return named;
+    if (env.TEWIP_RALLY_KEY)
+        return 'rally';
+    if (env.TEWIP_GITHUB_REPO && !env.TEWIP_JIRA_URL)
+        return 'github';
+    return 'jira';
+}
+export function trackerHelp() {
+    return [
+        'No tracker is set up. Tewip files work items into whichever of these you use:',
+        '',
+        '  Jira (Cloud, Server or Data Center)',
+        '    TEWIP_JIRA_URL=https://your-team.atlassian.net   TEWIP_JIRA_EMAIL=you@company.com   TEWIP_JIRA_TOKEN=...',
+        '',
+        '  Rally (Broadcom, formerly CA Agile Central)',
+        '    TEWIP_RALLY_KEY=_abc123...   TEWIP_RALLY_PROJECT=/project/12345   (TEWIP_RALLY_URL for on-premise)',
+        '',
+        '  GitHub Issues',
+        '    TEWIP_GITHUB_REPO=owner/name   TEWIP_GITHUB_TOKEN=ghp_...  (or be logged in with the gh CLI)',
+        '',
+        'Set TEWIP_TRACKER=jira|rally|github when more than one is configured.',
+    ].join('\n');
+}

package/dist/engine/work/trackers.js

@@ -0,0 +1,72 @@
+// Picking the tracker a team actually uses, and making Jira look like the others.
+//
+// Everything above this line in Tewip talks to `Tracker`, so adding Linear or Azure Boards later is
+// one file and no changes anywhere else - and the promises (create, never move, never assign, never
+// close, never file twice) hold for all of them by construction.
+import { Jira, jiraFromEnv } from './jira.js';
+import { Rally, rallyFromEnv } from './rally.js';
+import { GitHubIssues, githubIssuesFromEnv } from './github-issues.js';
+import { priorityWord, trackerHelp, trackerNameFromEnv } from './tracker.js';
+/** Jira, wearing the same interface as the rest. */
+export class JiraTracker {
+    name = 'Jira';
+    jira;
+    hasToken;
+    /** Issue types this project really has, read once. */
+    types = null;
+    constructor(jira, hasToken) {
+        this.jira = jira;
+        this.hasToken = hasToken;
+    }
+    canWrite() {
+        return this.hasToken;
+    }
+    async mine(project, label) {
+        return (await this.jira.findByLabel(project, label)).map((i) => ({ key: i.key, title: i.summary }));
+    }
+    /** Jira projects differ in which types exist, so the nearest one that does is used. */
+    async typeFor(project, kind) {
+        this.types ??= await this.jira.issueTypes(project);
+        const wanted = {
+            'test-case': ['Test', 'Test Case', 'Task'],
+            bug: ['Bug', 'Defect', 'Task'],
+            story: ['Story', 'User Story', 'Task'],
+            epic: ['Epic', 'Feature', 'Task'],
+        };
+        const found = wanted[kind].find((t) => this.types.includes(t)) ?? this.types[0];
+        if (!found)
+            throw new Error(`no issue type available in ${project}`);
+        return found;
+    }
+    async create(project, item) {
+        return this.jira.createIssue({
+            projectKey: project,
+            type: await this.typeFor(project, item.kind),
+            summary: item.title,
+            description: item.body,
+            labels: item.labels,
+            parentKey: item.parent,
+        });
+    }
+    async comment(key, text) {
+        await this.jira.comment(key, text);
+    }
+}
+/**
+ * The tracker the settings point at, or null when none is configured. A tracker that can only be
+ * read (a public Jira, or GitHub without a token) still comes back: a dry run needs no write.
+ */
+export function trackerFromEnv(projectKey = '', env = process.env) {
+    const id = trackerNameFromEnv(env);
+    if (id === 'rally') {
+        const config = rallyFromEnv(env);
+        return config ? { tracker: new Rally(config), project: projectKey || config.project, id } : null;
+    }
+    if (id === 'github') {
+        const config = githubIssuesFromEnv(env);
+        return config ? { tracker: new GitHubIssues(config), project: projectKey || config.repo, id } : null;
+    }
+    const config = jiraFromEnv();
+    return config ? { tracker: new JiraTracker(new Jira(config), Boolean(config.token)), project: projectKey, id } : null;
+}
+export { priorityWord, trackerHelp };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tewip-ai",
-  "version": "0.8.1",
+  "version": "0.10.0",
   "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
   "keywords": [
     "test-failures",