tewip-ai 0.10.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +30 -0
- package/TEAM-GUIDE.md +2 -1
- package/dist/cli/tewip.js +19 -1
- package/dist/engine/autofix/fix-selector.js +93 -0
- package/dist/engine/autofix/selector-source.js +162 -0
- package/dist/engine/import/junit.js +40 -4
- package/dist/engine/import/other-formats.js +127 -0
- package/dist/engine/review/coverage.js +188 -0
- package/dist/engine/review/review-command.js +44 -0
- package/dist/engine/review/review.js +127 -0
- package/dist/engine/work/azure-linear.js +116 -0
- package/dist/engine/work/tracker.js +14 -3
- package/dist/engine/work/trackers.js +9 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -166,6 +166,36 @@ that lost its markup) and **5 of 5 harmless changes ignored** (capitals, extra s
|
|
|
166
166
|
implemented as a link, reordered sections). No designer has judged the output yet, so the roadmap's
|
|
167
167
|
own metric is still open.
|
|
168
168
|
|
|
169
|
+
## More runners: Robot Framework, JMeter, k6 (new)
|
|
170
|
+
|
|
171
|
+
Tewip now reads **Robot Framework's output.xml**, **JMeter's .jtl** and a **k6 summary** as well as
|
|
172
|
+
JUnit XML, TRX, NUnit and Playwright's own reports. A load test does not fail test by test, so a
|
|
173
|
+
JMeter report becomes one line per request ("4 of 5 requests failed, an assertion failed") and a k6
|
|
174
|
+
run becomes one record per crossed threshold, with the number that crossed it.
|
|
175
|
+
|
|
176
|
+
Appium and Selenium need no importer: those suites report through Mocha, pytest or JUnit, which
|
|
177
|
+
Tewip already reads, and their own error messages ("Could not find a connected device", "stale
|
|
178
|
+
element reference") are already understood.
|
|
179
|
+
|
|
180
|
+
Each reader was checked against real reports published in public repositories, which is how three
|
|
181
|
+
defects were found before release - a Robot timestamp that is not a date, k6's numbers living under
|
|
182
|
+
`values`, and a JMeter line that read "mostly 200" when an assertion had failed.
|
|
183
|
+
|
|
184
|
+
## Before anyone reads the diff (new)
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
tewip review --diff main...HEAD
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
Which tests cover this change and **how** (they import the file, they import the package that
|
|
191
|
+
exports it, or their words merely match); which of them have flaked before, so a red run is not
|
|
192
|
+
blamed on your change; what has gone wrong in these areas before; what nothing tests at all; and the
|
|
193
|
+
command to run the telling tests first. From your project's own history, so it costs nothing.
|
|
194
|
+
|
|
195
|
+
Measured on **16 real commits** from two public projects, where the developers changed source *and*
|
|
196
|
+
its test: given only the source files, Tewip **named the right test in 16 of 16, ranked it first in
|
|
197
|
+
12**.
|
|
198
|
+
|
|
169
199
|
## Which flaky tests to quarantine, and what they cost (new)
|
|
170
200
|
|
|
171
201
|
```bash
|
package/TEAM-GUIDE.md
CHANGED
|
@@ -127,6 +127,7 @@ collects environment failures in one issue, and (optionally) opens a fix pull re
|
|
|
127
127
|
| `tewip spec "<feature>"` | An epic, its user stories and acceptance criteria, grounded in your code; files them in Jira with `--project PROJ --create` | Before the work is broken down |
|
|
128
128
|
| `tewip report report.xml --gate` | The CI command for GitLab, Jenkins, CircleCI, Azure, Bitbucket and the rest: diagnoses the run, comments on the GitLab merge request, and fails the build only on failures that need a person | Your CI is not GitHub Actions |
|
|
129
129
|
| `tewip watch --cmd "npm test"` | Runs your tests when you save a file and explains each failure straight away | While you are writing code |
|
|
130
|
+
| `tewip review --diff main...HEAD` | Which tests cover your change, which have flaked before, what nothing tests, and what to run first | Before you open a pull request |
|
|
130
131
|
| `tewip flaky --days 30` | Which tests to quarantine, which to fix, what they cost in CI minutes - and when a quarantined one can come back | Monday morning |
|
|
131
132
|
| `tewip note --days 7` | The weekly quality note for a lead, from runs already diagnosed (no model calls, free) | Monday morning |
|
|
132
133
|
| `tewip fix --code report.xml` | Opt-in: proposes the smallest change to application code for a real bug, and keeps it only if the failing test passes and the whole suite still passes | "It is a real bug and I want a starting patch" |
|
|
@@ -140,7 +141,7 @@ Notes that save you time:
|
|
|
140
141
|
- `write` never touches your application code. It only adds new test files, and you review and
|
|
141
142
|
commit them yourself.
|
|
142
143
|
- Everything it writes locally goes to the `.tewip` folder in your project.
|
|
143
|
-
- Work items go to **Jira, Rally
|
|
144
|
+
- Work items go to **Jira, Rally, GitHub Issues, Azure Boards or Linear** - set `TEWIP_TRACKER`, or just the credentials
|
|
144
145
|
of the one you use. GitHub Issues uses your existing `gh` login, so it needs no new secret.
|
|
145
146
|
- Jira works with **Cloud, Server and Data Center**. To read tickets from a tracker anyone can read
|
|
146
147
|
(many open-source ones), `TEWIP_JIRA_URL` alone is enough - no account. Writing needs a token.
|
package/dist/cli/tewip.js
CHANGED
|
@@ -67,6 +67,12 @@ Usage:
|
|
|
67
67
|
the GitLab merge request, tells Slack or Teams only when a
|
|
68
68
|
failure needs a person, and with --gate fails the build only on
|
|
69
69
|
those failures.
|
|
70
|
+
tewip review [--diff main...HEAD] [--out review.md]
|
|
71
|
+
Before anyone reads the diff: which tests cover this change,
|
|
72
|
+
which of them have flaked before, what has gone wrong in these
|
|
73
|
+
areas, what nothing tests at all, and the command that runs the
|
|
74
|
+
telling tests first. From this project's own history, so it
|
|
75
|
+
costs nothing.
|
|
70
76
|
tewip watch [--cmd "npm test"] [--report <path>] [--paths src,tests]
|
|
71
77
|
Beside your editor: runs your tests when a file changes (or
|
|
72
78
|
watches for new reports) and explains each failure as it
|
|
@@ -92,7 +98,9 @@ Usage:
|
|
|
92
98
|
tewip --version Print the version.
|
|
93
99
|
|
|
94
100
|
Reports it reads: Playwright, pytest/unittest, Jest, Vitest, Mocha, node:test, JUnit/TestNG
|
|
95
|
-
(Maven, Gradle), Go (gotestsum), .NET (TRX or JUnit), RSpec, PHPUnit, Rust (nextest)
|
|
101
|
+
(Maven, Gradle), Go (gotestsum), .NET (TRX or JUnit), NUnit 3, RSpec, PHPUnit, Rust (nextest),
|
|
102
|
+
Robot Framework (output.xml), JMeter (.jtl) and k6 (--summary-export). Appium, Selenium and
|
|
103
|
+
WebdriverIO suites report through one of the above, and their own error messages are understood.
|
|
96
104
|
Set OPENAI_API_KEY to let the AI layer decide what the rules leave open.`;
|
|
97
105
|
const flag = (args, name) => {
|
|
98
106
|
const i = args.indexOf(`--${name}`);
|
|
@@ -248,6 +256,16 @@ async function main() {
|
|
|
248
256
|
process.exitCode = 1;
|
|
249
257
|
return;
|
|
250
258
|
}
|
|
259
|
+
case 'review': {
|
|
260
|
+
const { runReview } = await import('../engine/review/review-command.js');
|
|
261
|
+
await runReview({
|
|
262
|
+
root: process.cwd(),
|
|
263
|
+
diff: flag(args, 'diff') ?? 'main...HEAD',
|
|
264
|
+
out: flag(args, 'out'),
|
|
265
|
+
name: flag(args, 'name'),
|
|
266
|
+
});
|
|
267
|
+
return;
|
|
268
|
+
}
|
|
251
269
|
case 'watch': {
|
|
252
270
|
const { runWatch } = await import('../engine/watch/watch-command.js');
|
|
253
271
|
await runWatch({
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
// ACT + VERIFY for a drifted selector in a Cypress, Selenium or WebdriverIO test.
|
|
2
|
+
//
|
|
3
|
+
// The same loop as every other fix Tewip makes: change one thing in the test file, run that test,
|
|
4
|
+
// keep the change only if it passes, and otherwise put the file back exactly as it was. What is new
|
|
5
|
+
// is where the replacement comes from - the repository, not a page snapshot - which is why these
|
|
6
|
+
// tools can be fixed at all without a trace.
|
|
7
|
+
//
|
|
8
|
+
// It will not touch application code, will not change what the test asserts, and will not offer a
|
|
9
|
+
// replacement it cannot point at in the source or in a commit.
|
|
10
|
+
import { relative, resolve, sep } from 'node:path';
|
|
11
|
+
import { driverLocator } from '../driver-errors.js';
|
|
12
|
+
import { likeItInSource, renamedInGit, tokenOf, writeSelector } from './selector-source.js';
|
|
13
|
+
/**
|
|
14
|
+
* Where a selector is written in a test: the whole line, so the replacement reads in context.
|
|
15
|
+
* Cypress and Selenium put the selector in a plain string, so this is simpler than Playwright's
|
|
16
|
+
* chained calls - and looking for the quoted form keeps it from matching a comment about it.
|
|
17
|
+
*/
|
|
18
|
+
export function findSelectorSites(source, selector) {
|
|
19
|
+
const quoted = [`'${selector}'`, `"${selector}"`, `\`${selector}\``];
|
|
20
|
+
const sites = [];
|
|
21
|
+
source.split('\n').forEach((text, index) => {
|
|
22
|
+
if (quoted.some((q) => text.includes(q)))
|
|
23
|
+
sites.push({ line: index + 1, text });
|
|
24
|
+
});
|
|
25
|
+
return sites;
|
|
26
|
+
}
|
|
27
|
+
/**
|
|
28
|
+
* One drifted selector, fixed from what the repository knows. `keep` false checks a suggestion
|
|
29
|
+
* without leaving it in the file, exactly as the locator fix does.
|
|
30
|
+
*/
|
|
31
|
+
export async function attemptSelectorFix({ record, diagnosis, action, specFile }, deps, { keep = true } = {}) {
|
|
32
|
+
const title = record.testId.split(' > ').at(-1) ?? record.testId;
|
|
33
|
+
const result = { testId: record.testId, specFile, status: 'not-eligible', applied: false, log: [] };
|
|
34
|
+
const stop = (status, why) => ({ ...result, status, log: [...result.log, why] });
|
|
35
|
+
if (diagnosis.category !== 'LOCATOR_DRIFT' || action.kind !== 'fix-locator') {
|
|
36
|
+
return stop('not-eligible', `only a renamed element is fixed this way; this one is ${diagnosis.category} (${action.kind})`);
|
|
37
|
+
}
|
|
38
|
+
const rel = relative(resolve(deps.allowedRoot), resolve(specFile));
|
|
39
|
+
if (rel.startsWith('..') || rel.startsWith(sep) || /^[a-zA-Z]:/.test(rel))
|
|
40
|
+
return stop('outside-sandbox', `refused: ${specFile} is outside ${deps.allowedRoot}`);
|
|
41
|
+
// What the tool said it was looking for.
|
|
42
|
+
const selector = driverLocator(record.errorMessage) ?? '';
|
|
43
|
+
if (!selector)
|
|
44
|
+
return stop('no-selector', 'the error does not say which selector was not found');
|
|
45
|
+
result.log.push(`the test looked for ${selector}`);
|
|
46
|
+
// What the line in the test really holds: Selenium prints "css selector=#x" but writes "#x".
|
|
47
|
+
const written = /^[a-z ]+=\s*/i.test(selector) ? selector.replace(/^[a-z ]+=\s*/i, '') : selector;
|
|
48
|
+
const token = tokenOf(selector);
|
|
49
|
+
if (!token) {
|
|
50
|
+
return stop('not-identifiable', `${selector} is not an id, class or data attribute, so there is nothing in the source to compare it against. A person should look at the page.`);
|
|
51
|
+
}
|
|
52
|
+
// Git first: somebody renamed it, and the commit says to what. The source is the weaker answer.
|
|
53
|
+
const found = renamedInGit(deps.projectRoot, token) ?? likeItInSource(deps.projectRoot, token, deps.sources);
|
|
54
|
+
if (!found) {
|
|
55
|
+
return stop('no-replacement', `nothing in the source or in the history looks like "${token.value}" renamed. Tewip will not invent a selector.`);
|
|
56
|
+
}
|
|
57
|
+
const evidence = found.how === 'renamed in git'
|
|
58
|
+
? `commit ${found.commit} changed "${token.value}" to "${found.value}" in ${found.file}`
|
|
59
|
+
: `"${token.value}" is nowhere in the source any more, and ${found.file} now has "${found.value}"`;
|
|
60
|
+
result.evidence = evidence;
|
|
61
|
+
result.log.push(evidence);
|
|
62
|
+
const replacement = writeSelector(token, found.value);
|
|
63
|
+
const source = deps.readFile(specFile);
|
|
64
|
+
const sites = findSelectorSites(source, written);
|
|
65
|
+
const site = sites[0];
|
|
66
|
+
if (!site)
|
|
67
|
+
return stop('selector-not-in-test', `${selector} is not written in ${rel}: it comes from a helper or a page object, which a person should change`);
|
|
68
|
+
result.line = site.line;
|
|
69
|
+
result.original = site.text;
|
|
70
|
+
result.replacement = site.text.replace(written, replacement);
|
|
71
|
+
deps.writeFile(specFile, source.replace(site.text, result.replacement));
|
|
72
|
+
try {
|
|
73
|
+
const run = await deps.verify(specFile, title);
|
|
74
|
+
if (!run.passed) {
|
|
75
|
+
deps.writeFile(specFile, source);
|
|
76
|
+
return stop('still-failing', `with ${replacement} the test still failed, so the file was put back: ${run.detail.slice(-200)}`);
|
|
77
|
+
}
|
|
78
|
+
result.log.push(`with ${replacement} the test passes`);
|
|
79
|
+
if (!keep)
|
|
80
|
+
deps.writeFile(specFile, source);
|
|
81
|
+
return { ...result, status: 'fixed', applied: keep };
|
|
82
|
+
}
|
|
83
|
+
catch (error) {
|
|
84
|
+
deps.writeFile(specFile, source);
|
|
85
|
+
throw error;
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
/** The one-line diff a reviewer reads. */
|
|
89
|
+
export function selectorFixDiff(result) {
|
|
90
|
+
if (!result.original || !result.replacement)
|
|
91
|
+
return '';
|
|
92
|
+
return [`--- a/${result.specFile}`, `+++ b/${result.specFile}`, `@@ line ${result.line} @@`, `-${result.original.trim()}`, `+${result.replacement.trim()}`].join('\n');
|
|
93
|
+
}
|
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
// Phase 5, for the tools that do not have a page snapshot: Cypress, Selenium and WebdriverIO.
|
|
2
|
+
//
|
|
3
|
+
// Playwright's traces tell Tewip what was on the page, so a drifted locator can be matched against
|
|
4
|
+
// the real elements. A Cypress or Selenium failure in a JUnit report carries none of that - only
|
|
5
|
+
// "Expected to find element: '#submit-btn', but never found it". Inventing a replacement from that
|
|
6
|
+
// would be exactly the kind of guess this project exists not to make.
|
|
7
|
+
//
|
|
8
|
+
// So the answer comes from the repository instead. If a test looks for `#submit-btn` and the app
|
|
9
|
+
// now renders `id="submit-button"`, the new name is in the application source, and git knows which
|
|
10
|
+
// commit renamed it. That is evidence, not a guess - and the fix is still only kept if the test
|
|
11
|
+
// passes afterwards.
|
|
12
|
+
import { execFileSync } from 'node:child_process';
|
|
13
|
+
import { readFileSync } from 'node:fs';
|
|
14
|
+
/** What a CSS or XPath selector is really identifying, when it is something identifiable. */
|
|
15
|
+
export function tokenOf(selector) {
|
|
16
|
+
// Selenium and WebdriverIO name the strategy first: "css selector=#submit-btn", "id=submit-btn".
|
|
17
|
+
const strategy = /^(css selector|css|xpath|id|name|class name|link text|partial link text|tag name)\s*=\s*(.+)$/i.exec(selector.trim());
|
|
18
|
+
const s = (strategy ? strategy[2] : selector).trim();
|
|
19
|
+
if (strategy) {
|
|
20
|
+
const method = strategy[1].toLowerCase();
|
|
21
|
+
if (method === 'id')
|
|
22
|
+
return { kind: 'id', value: s };
|
|
23
|
+
if (method === 'class name')
|
|
24
|
+
return { kind: 'class', value: s.split(/\s+/)[0] };
|
|
25
|
+
if (method === 'link text' || method === 'partial link text')
|
|
26
|
+
return { kind: 'text', value: s };
|
|
27
|
+
}
|
|
28
|
+
const data = /\[\s*([\w-]*data[\w-]*)\s*[~|^$*]?=\s*["']?([^"'\]]+)["']?\s*\]/i.exec(s);
|
|
29
|
+
if (data)
|
|
30
|
+
return { kind: 'data', attribute: data[1], value: data[2] };
|
|
31
|
+
const id = /^#([\w-]+)$|\[\s*id\s*=\s*["']([^"']+)["']\s*\]|@id\s*=\s*["']([^"']+)["']/i.exec(s);
|
|
32
|
+
if (id)
|
|
33
|
+
return { kind: 'id', value: id[1] ?? id[2] ?? id[3] };
|
|
34
|
+
const cls = /^\.([\w-]+)$|\[\s*class\s*=\s*["']([^"']+)["']\s*\]/i.exec(s);
|
|
35
|
+
if (cls)
|
|
36
|
+
return { kind: 'class', value: (cls[1] ?? cls[2]).split(/\s+/)[0] };
|
|
37
|
+
const text = /text\(\)\s*=\s*["']([^"']+)["']|contains\(\s*(?:text\(\)|\.)\s*,\s*["']([^"']+)["']/i.exec(s);
|
|
38
|
+
if (text)
|
|
39
|
+
return { kind: 'text', value: text[1] ?? text[2] };
|
|
40
|
+
return null;
|
|
41
|
+
}
|
|
42
|
+
/** How close two names are, 0 to 1, by the words in them - "submit-btn" and "submit-button". */
|
|
43
|
+
export function closeness(a, b) {
|
|
44
|
+
const words = (s) => new Set(s.toLowerCase().split(/[^a-z0-9]+/).filter(Boolean));
|
|
45
|
+
const left = words(a);
|
|
46
|
+
const right = words(b);
|
|
47
|
+
if (left.size === 0 || right.size === 0)
|
|
48
|
+
return 0;
|
|
49
|
+
let shared = 0;
|
|
50
|
+
for (const w of left)
|
|
51
|
+
if (right.has(w))
|
|
52
|
+
shared++;
|
|
53
|
+
// A shared prefix counts too: "submitBtn" and "submitButton" have no word in common.
|
|
54
|
+
const prefix = commonPrefix(a.toLowerCase(), b.toLowerCase()) / Math.max(a.length, b.length);
|
|
55
|
+
return Math.max(shared / Math.max(left.size, right.size), prefix);
|
|
56
|
+
}
|
|
57
|
+
function commonPrefix(a, b) {
|
|
58
|
+
let i = 0;
|
|
59
|
+
while (i < a.length && i < b.length && a[i] === b[i])
|
|
60
|
+
i++;
|
|
61
|
+
return i;
|
|
62
|
+
}
|
|
63
|
+
const git = (root, args) => {
|
|
64
|
+
try {
|
|
65
|
+
return execFileSync('git', args, { cwd: root, encoding: 'utf8', maxBuffer: 32 * 1024 * 1024, timeout: 60_000 });
|
|
66
|
+
}
|
|
67
|
+
catch {
|
|
68
|
+
return '';
|
|
69
|
+
}
|
|
70
|
+
};
|
|
71
|
+
/**
|
|
72
|
+
* The commit that removed this value, and what took its place on the same line. This is the
|
|
73
|
+
* strongest evidence there is: somebody renamed it, and the diff says to what.
|
|
74
|
+
*/
|
|
75
|
+
export function renamedInGit(root, token, maxCommits = 5) {
|
|
76
|
+
const shas = git(root, ['log', '-S', token.value, '--format=%H', `-n${maxCommits}`, '--', '.'])
|
|
77
|
+
.split('\n')
|
|
78
|
+
.map((s) => s.trim())
|
|
79
|
+
.filter(Boolean);
|
|
80
|
+
for (const sha of shas) {
|
|
81
|
+
const diff = git(root, ['show', '--unified=0', '--format=', sha]);
|
|
82
|
+
if (!diff)
|
|
83
|
+
continue;
|
|
84
|
+
let file = '';
|
|
85
|
+
const removed = [];
|
|
86
|
+
const added = [];
|
|
87
|
+
for (const line of diff.split('\n')) {
|
|
88
|
+
if (line.startsWith('+++ b/'))
|
|
89
|
+
file = line.slice(6);
|
|
90
|
+
else if (line.startsWith('-') && line.includes(token.value))
|
|
91
|
+
removed.push(line);
|
|
92
|
+
else if (line.startsWith('+'))
|
|
93
|
+
added.push(line);
|
|
94
|
+
}
|
|
95
|
+
if (removed.length === 0 || added.length === 0)
|
|
96
|
+
continue;
|
|
97
|
+
// The best replacement is the attribute value on an added line that most resembles the old one.
|
|
98
|
+
let best = null;
|
|
99
|
+
for (const line of added) {
|
|
100
|
+
for (const value of attributeValues(line, token)) {
|
|
101
|
+
const score = closeness(token.value, value);
|
|
102
|
+
if (value !== token.value && score >= 0.5 && (!best || score > best.score)) {
|
|
103
|
+
best = { value, file, how: 'renamed in git', commit: sha.slice(0, 8), score };
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
if (best)
|
|
108
|
+
return best;
|
|
109
|
+
}
|
|
110
|
+
return null;
|
|
111
|
+
}
|
|
112
|
+
/** Values of the same kind of attribute written on a line of source. */
|
|
113
|
+
function attributeValues(line, token) {
|
|
114
|
+
const out = [];
|
|
115
|
+
const attribute = token.kind === 'data' ? token.attribute ?? 'data-[\\w-]+' : token.kind === 'id' ? 'id' : token.kind === 'class' ? 'class(?:Name)?' : '';
|
|
116
|
+
if (!attribute)
|
|
117
|
+
return out;
|
|
118
|
+
const pattern = new RegExp(`${attribute}\\s*=\\s*["'\`{]?\\s*["']?([\\w\\s-]+)["']?`, 'gi');
|
|
119
|
+
for (const m of line.matchAll(pattern)) {
|
|
120
|
+
for (const value of m[1].trim().split(/\s+/))
|
|
121
|
+
if (value)
|
|
122
|
+
out.push(value);
|
|
123
|
+
}
|
|
124
|
+
return out;
|
|
125
|
+
}
|
|
126
|
+
/**
|
|
127
|
+
* The closest value of the same kind anywhere in the application source now. Weaker than git,
|
|
128
|
+
* because nobody wrote down that it is a rename - so it is offered only when it is clearly close.
|
|
129
|
+
*/
|
|
130
|
+
export function likeItInSource(root, token, sources, min = 0.6) {
|
|
131
|
+
let best = null;
|
|
132
|
+
for (const file of sources) {
|
|
133
|
+
let text;
|
|
134
|
+
try {
|
|
135
|
+
text = readFileSync(file, 'utf8');
|
|
136
|
+
}
|
|
137
|
+
catch {
|
|
138
|
+
continue;
|
|
139
|
+
}
|
|
140
|
+
if (text.includes(token.value))
|
|
141
|
+
return null; // it is still there; this is not a rename
|
|
142
|
+
for (const line of text.split('\n')) {
|
|
143
|
+
for (const value of attributeValues(line, token)) {
|
|
144
|
+
const score = closeness(token.value, value);
|
|
145
|
+
if (value !== token.value && score >= min && (!best || score > best.score)) {
|
|
146
|
+
best = { value, file, how: 'found in the source', score };
|
|
147
|
+
}
|
|
148
|
+
}
|
|
149
|
+
}
|
|
150
|
+
}
|
|
151
|
+
return best;
|
|
152
|
+
}
|
|
153
|
+
/** The selector written the way each tool writes it, so the fix reads like the rest of the test. */
|
|
154
|
+
export function writeSelector(token, value) {
|
|
155
|
+
if (token.kind === 'id')
|
|
156
|
+
return `#${value}`;
|
|
157
|
+
if (token.kind === 'class')
|
|
158
|
+
return `.${value}`;
|
|
159
|
+
if (token.kind === 'data')
|
|
160
|
+
return `[${token.attribute}="${value}"]`;
|
|
161
|
+
return value;
|
|
162
|
+
}
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
// Imports test reports from any language or framework into the agent's records: JUnit XML, which
|
|
2
2
|
// pytest, Jest, Vitest, Mocha, node:test, Maven Surefire, Gradle, gotestsum, PHPUnit, RSpec,
|
|
3
|
-
// cargo-nextest and Cypress write
|
|
3
|
+
// cargo-nextest and Cypress write; .NET's own TRX; NUnit 3's result file; Robot Framework's
|
|
4
|
+
// output.xml; JMeter's .jtl; and a k6 summary, where a run fails by crossing a threshold rather
|
|
5
|
+
// than test by test. Playwright keeps its own importer
|
|
4
6
|
// (blob.ts), which also reads traces.
|
|
5
7
|
//
|
|
6
8
|
// What these reports can and cannot say: each test's name, outcome, failure message and stack,
|
|
@@ -11,6 +13,7 @@ import { readdirSync, readFileSync, statSync } from 'node:fs';
|
|
|
11
13
|
import { join } from 'node:path';
|
|
12
14
|
import { SCHEMA_VERSION } from '../../reporters/types.js';
|
|
13
15
|
import { descendants, parseXml } from './xml.js';
|
|
16
|
+
import { jmeterCases, k6Cases, looksLikeK6, robotCases, robotTime } from './other-formats.js';
|
|
14
17
|
const PASSED = { failed: false, message: '', stack: '' };
|
|
15
18
|
const FAILURE = new Set(['failure', 'error']);
|
|
16
19
|
/** Maven Surefire's records of earlier attempts: flaky* when a later one passed, rerun* when not. */
|
|
@@ -138,7 +141,8 @@ function nunitCases(root) {
|
|
|
138
141
|
};
|
|
139
142
|
});
|
|
140
143
|
}
|
|
141
|
-
|
|
144
|
+
// .jtl is JMeter's, and a k6 summary is JSON; both are read below.
|
|
145
|
+
const REPORT_FILE = /\.(xml|trx|jtl|json)$/i;
|
|
142
146
|
/** Report files under `path` (a file, or a directory searched recursively). */
|
|
143
147
|
export function junitFiles(path) {
|
|
144
148
|
if (!statSync(path).isDirectory())
|
|
@@ -162,7 +166,26 @@ export function importJunit(path, meta) {
|
|
|
162
166
|
const unread = [];
|
|
163
167
|
let parsedFiles = 0;
|
|
164
168
|
for (const file of junitFiles(path)) {
|
|
165
|
-
const
|
|
169
|
+
const raw = readFileSync(file, 'utf8');
|
|
170
|
+
// k6 writes JSON, not XML: a performance run fails by crossing a threshold, not test by test.
|
|
171
|
+
if (/\.json$/i.test(file)) {
|
|
172
|
+
let parsed;
|
|
173
|
+
try {
|
|
174
|
+
parsed = JSON.parse(raw);
|
|
175
|
+
}
|
|
176
|
+
catch {
|
|
177
|
+
unread.push(`${file} (not JSON)`);
|
|
178
|
+
continue;
|
|
179
|
+
}
|
|
180
|
+
if (!looksLikeK6(parsed)) {
|
|
181
|
+
unread.push(`${file} (JSON, but not a k6 summary)`);
|
|
182
|
+
continue;
|
|
183
|
+
}
|
|
184
|
+
parsedFiles++;
|
|
185
|
+
addCases(k6Cases(parsed));
|
|
186
|
+
continue;
|
|
187
|
+
}
|
|
188
|
+
const root = parseXml(raw);
|
|
166
189
|
let cases;
|
|
167
190
|
if (root.name === 'TestRun') {
|
|
168
191
|
cases = trxCases(root);
|
|
@@ -182,6 +205,15 @@ export function importJunit(path, meta) {
|
|
|
182
205
|
if (start && (!startedAt || start < startedAt))
|
|
183
206
|
startedAt = start;
|
|
184
207
|
}
|
|
208
|
+
else if (root.name === 'robot') {
|
|
209
|
+
cases = robotCases(root);
|
|
210
|
+
const start = robotTime(root.attrs.generated ?? '');
|
|
211
|
+
if (start && !startedAt)
|
|
212
|
+
startedAt = start;
|
|
213
|
+
}
|
|
214
|
+
else if (root.name === 'testResults') {
|
|
215
|
+
cases = jmeterCases(root);
|
|
216
|
+
}
|
|
185
217
|
else {
|
|
186
218
|
// Not a report Tewip knows. Staying silent here would report a failing run as clean, so the
|
|
187
219
|
// file is remembered and, if nothing at all could be read, the caller is told.
|
|
@@ -189,6 +221,9 @@ export function importJunit(path, meta) {
|
|
|
189
221
|
continue;
|
|
190
222
|
}
|
|
191
223
|
parsedFiles++;
|
|
224
|
+
addCases(cases);
|
|
225
|
+
}
|
|
226
|
+
function addCases(cases) {
|
|
192
227
|
for (const c of cases) {
|
|
193
228
|
attempts[c.testId] = (attempts[c.testId] ?? 0) + c.attempts.length;
|
|
194
229
|
c.attempts.forEach((attempt, retry) => {
|
|
@@ -226,7 +261,8 @@ export function importJunit(path, meta) {
|
|
|
226
261
|
if (unread.length > 0) {
|
|
227
262
|
const list = unread.slice(0, 5).join(', ');
|
|
228
263
|
if (parsedFiles === 0) {
|
|
229
|
-
throw new Error(`no test report Tewip can read in ${path}: ${list}. It reads JUnit XML (<testsuites>/<testsuite>), NUnit 3 (<test-run>)
|
|
264
|
+
throw new Error(`no test report Tewip can read in ${path}: ${list}. It reads JUnit XML (<testsuites>/<testsuite>), NUnit 3 (<test-run>), .NET TRX (<TestRun>), ` +
|
|
265
|
+
'Robot Framework output.xml (<robot>), JMeter .jtl (<testResults>) and a k6 summary (--summary-export). ' +
|
|
230
266
|
'Most runners can write JUnit XML; for TestNG use the Surefire report, and for NUnit use --result=x.xml.');
|
|
231
267
|
}
|
|
232
268
|
console.error(`[tewip] ignored ${unread.length} file(s) that are not a test report: ${list}`);
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
import { descendants } from './xml.js';
|
|
2
|
+
const text = (element) => (element ? element.text.trim() : '');
|
|
3
|
+
/**
|
|
4
|
+
* Robot Framework writes output.xml by default, and only writes JUnit if asked. A suite holds
|
|
5
|
+
* tests, a test holds a status with the failure message inside it, and suites nest - so the test's
|
|
6
|
+
* name is its whole path, the way Robot itself prints it.
|
|
7
|
+
*/
|
|
8
|
+
export function robotCases(root) {
|
|
9
|
+
const cases = [];
|
|
10
|
+
const walk = (element, path, source) => {
|
|
11
|
+
for (const child of element.children) {
|
|
12
|
+
if (child.name === 'suite') {
|
|
13
|
+
walk(child, [...path, child.attrs.name ?? ''], child.attrs.source ?? source);
|
|
14
|
+
continue;
|
|
15
|
+
}
|
|
16
|
+
if (child.name !== 'test')
|
|
17
|
+
continue;
|
|
18
|
+
const status = child.children.find((c) => c.name === 'status');
|
|
19
|
+
const failed = (status?.attrs.status ?? '').toUpperCase() === 'FAIL';
|
|
20
|
+
const start = status?.attrs.starttime ?? status?.attrs.start ?? '';
|
|
21
|
+
const end = status?.attrs.endtime ?? status?.attrs.end ?? '';
|
|
22
|
+
// The last keyword that failed says where it went wrong, which the message alone does not.
|
|
23
|
+
const failedKeyword = descendants(child, 'kw').find((kw) => (kw.children.find((c) => c.name === 'status')?.attrs.status ?? '').toUpperCase() === 'FAIL');
|
|
24
|
+
const where = failedKeyword?.attrs.name ? `Keyword: ${failedKeyword.attrs.name}` : '';
|
|
25
|
+
cases.push({
|
|
26
|
+
testId: [...path, child.attrs.name ?? ''].filter(Boolean).join(' > '),
|
|
27
|
+
file: source,
|
|
28
|
+
duration: robotDuration(start, end),
|
|
29
|
+
attempts: [{ failed, message: failed ? text(status) || 'the test failed' : '', stack: failed ? where : '' }],
|
|
30
|
+
});
|
|
31
|
+
}
|
|
32
|
+
};
|
|
33
|
+
walk(root, [], root.attrs.source ?? '');
|
|
34
|
+
return cases;
|
|
35
|
+
}
|
|
36
|
+
/** Robot's timestamp as an ISO one, or '' when it is not the shape Robot writes. */
|
|
37
|
+
export function robotTime(stamp) {
|
|
38
|
+
const at = robotStamp(stamp);
|
|
39
|
+
return at > 0 ? new Date(at).toISOString() : '';
|
|
40
|
+
}
|
|
41
|
+
/** Robot's own timestamp shape: 20260923 14:05:01.123 */
|
|
42
|
+
function robotStamp(s) {
|
|
43
|
+
const m = /^(\d{4})(\d{2})(\d{2}) (\d{2}):(\d{2}):(\d{2})\.(\d{1,3})$/.exec(s.trim());
|
|
44
|
+
return m ? Date.UTC(Number(m[1]), Number(m[2]) - 1, Number(m[3]), Number(m[4]), Number(m[5]), Number(m[6]), Number(m[7])) : 0;
|
|
45
|
+
}
|
|
46
|
+
function robotDuration(start, end) {
|
|
47
|
+
const from = robotStamp(start);
|
|
48
|
+
const to = robotStamp(end);
|
|
49
|
+
return from > 0 && to >= from ? to - from : 0;
|
|
50
|
+
}
|
|
51
|
+
/**
|
|
52
|
+
* JMeter's .jtl in XML form: one sample per request, s="false" when it failed. A load test has
|
|
53
|
+
* thousands of samples, so they are grouped by label - nobody wants ten thousand records, they want
|
|
54
|
+
* "the checkout request failed 412 times, mostly 503".
|
|
55
|
+
*/
|
|
56
|
+
export function jmeterCases(root) {
|
|
57
|
+
const samples = [...descendants(root, 'httpSample'), ...descendants(root, 'sample')];
|
|
58
|
+
const byLabel = new Map();
|
|
59
|
+
for (const sample of samples) {
|
|
60
|
+
const label = sample.attrs.lb ?? 'sample';
|
|
61
|
+
const found = byLabel.get(label) ?? { failed: 0, total: 0, time: 0, codes: new Map(), messages: new Set() };
|
|
62
|
+
found.total++;
|
|
63
|
+
found.time += Number(sample.attrs.t ?? 0);
|
|
64
|
+
if ((sample.attrs.s ?? 'true').toLowerCase() === 'false') {
|
|
65
|
+
found.failed++;
|
|
66
|
+
const code = `${sample.attrs.rc ?? ''} ${sample.attrs.rm ?? ''}`.trim();
|
|
67
|
+
if (code)
|
|
68
|
+
found.codes.set(code, (found.codes.get(code) ?? 0) + 1);
|
|
69
|
+
const assertion = descendants(sample, 'failureMessage')[0];
|
|
70
|
+
if (assertion)
|
|
71
|
+
found.messages.add(text(assertion).slice(0, 200));
|
|
72
|
+
}
|
|
73
|
+
byLabel.set(label, found);
|
|
74
|
+
}
|
|
75
|
+
return [...byLabel].map(([label, s]) => {
|
|
76
|
+
const worst = [...s.codes].sort((a, b) => b[1] - a[1])[0];
|
|
77
|
+
// A 2xx that still counts as failed is an assertion failing, not the server: "mostly 200" would
|
|
78
|
+
// read as nonsense to whoever opens the report.
|
|
79
|
+
const why = worst ? (/^2\d\d/.test(worst[0]) ? `, an assertion failed (${worst[1]} times, response ${worst[0]})` : `, mostly ${worst[0]} (${worst[1]} times)`) : '';
|
|
80
|
+
const message = s.failed === 0 ? '' : `${s.failed} of ${s.total} requests failed${why}${s.messages.size > 0 ? `: ${[...s.messages][0]}` : ''}`;
|
|
81
|
+
return { testId: `JMeter > ${label}`, file: '', duration: s.time, attempts: [{ failed: s.failed > 0, message, stack: [...s.messages].join('\n').slice(0, 2000) }] };
|
|
82
|
+
});
|
|
83
|
+
}
|
|
84
|
+
/**
|
|
85
|
+
* k6 writes a summary as JSON (`k6 run --summary-export=summary.json`). A performance run does not
|
|
86
|
+
* fail test by test: it fails when a threshold is crossed, and that is what is imported - one
|
|
87
|
+
* record per crossed threshold, saying by how much.
|
|
88
|
+
*/
|
|
89
|
+
export function k6Cases(json) {
|
|
90
|
+
const report = json;
|
|
91
|
+
if (!report?.metrics)
|
|
92
|
+
return [];
|
|
93
|
+
const cases = [];
|
|
94
|
+
for (const [metric, body] of Object.entries(report.metrics)) {
|
|
95
|
+
const thresholds = body.thresholds;
|
|
96
|
+
if (!thresholds)
|
|
97
|
+
continue;
|
|
98
|
+
for (const [expression, result] of Object.entries(thresholds)) {
|
|
99
|
+
// k6 writes either a boolean "ok" or an object with it; a crossed threshold is ok === false.
|
|
100
|
+
const ok = typeof result === 'boolean' ? result : Boolean(result?.ok);
|
|
101
|
+
const measured = describeMetric((body.values ?? body));
|
|
102
|
+
cases.push({
|
|
103
|
+
testId: `k6 > ${metric} > ${expression}`,
|
|
104
|
+
file: '',
|
|
105
|
+
duration: 0,
|
|
106
|
+
attempts: [{ failed: !ok, message: ok ? '' : `the threshold "${expression}" on ${metric} was crossed${measured ? `: ${measured}` : ''}`, stack: measured }],
|
|
107
|
+
});
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
return cases;
|
|
111
|
+
}
|
|
112
|
+
/** The numbers k6 reports for a metric, in one readable line. */
|
|
113
|
+
function describeMetric(body) {
|
|
114
|
+
const wanted = ['avg', 'min', 'med', 'max', 'p(90)', 'p(95)', 'p(99)', 'rate', 'count', 'value'];
|
|
115
|
+
return wanted
|
|
116
|
+
.filter((key) => typeof body[key] === 'number')
|
|
117
|
+
.map((key) => `${key}=${round(body[key])}`)
|
|
118
|
+
.join(' ');
|
|
119
|
+
}
|
|
120
|
+
const round = (n) => (Number.isInteger(n) ? String(n) : n.toFixed(2));
|
|
121
|
+
/** Whether a parsed JSON file looks like a k6 summary, so nothing else is read as one. */
|
|
122
|
+
export function looksLikeK6(json) {
|
|
123
|
+
const report = json;
|
|
124
|
+
if (!report?.metrics || typeof report.metrics !== 'object')
|
|
125
|
+
return false;
|
|
126
|
+
return ['http_req_duration', 'iterations', 'vus', 'checks', 'http_reqs'].some((m) => m in report.metrics);
|
|
127
|
+
}
|
|
@@ -0,0 +1,188 @@
|
|
|
1
|
+
// Which tests actually reach the code a change touched.
|
|
2
|
+
//
|
|
3
|
+
// Two ways of knowing, in this order: a test that imports the changed file (directly, or through
|
|
4
|
+
// the package's front door) really does exercise it; a test whose name and path share the change's
|
|
5
|
+
// words probably does. The first is a fact, the second a guess, and the report says which is which -
|
|
6
|
+
// because "these 6 tests cover your change" is only useful if it is true.
|
|
7
|
+
import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs';
|
|
8
|
+
import { basename, dirname, extname, join, relative, resolve, sep } from 'node:path';
|
|
9
|
+
const TEST_FILE = /(?:\.|_)(?:spec|test)\.[cm]?[jt]sx?$|(?:^|[\\/])test_[^\\/]+\.py$|_test\.py$|_test\.go$|Test(?:s)?\.(?:java|kt|cs)$|\.cy\.[jt]sx?$|\.feature$|\.robot$/i;
|
|
10
|
+
const SKIP_DIR = new Set(['node_modules', '.git', 'dist', 'build', 'out', 'coverage', 'target', 'vendor', '__pycache__', '.venv', 'venv', '.tewip', '.next']);
|
|
11
|
+
/** Every test file in the project. */
|
|
12
|
+
export function testFiles(root, limit = 4000) {
|
|
13
|
+
const out = [];
|
|
14
|
+
const walk = (dir) => {
|
|
15
|
+
if (out.length >= limit)
|
|
16
|
+
return;
|
|
17
|
+
let entries;
|
|
18
|
+
try {
|
|
19
|
+
entries = readdirSync(dir);
|
|
20
|
+
}
|
|
21
|
+
catch {
|
|
22
|
+
return;
|
|
23
|
+
}
|
|
24
|
+
for (const entry of entries) {
|
|
25
|
+
const path = join(dir, entry);
|
|
26
|
+
let info;
|
|
27
|
+
try {
|
|
28
|
+
info = statSync(path);
|
|
29
|
+
}
|
|
30
|
+
catch {
|
|
31
|
+
continue;
|
|
32
|
+
}
|
|
33
|
+
if (info.isDirectory()) {
|
|
34
|
+
if (!SKIP_DIR.has(entry))
|
|
35
|
+
walk(path);
|
|
36
|
+
}
|
|
37
|
+
else if (TEST_FILE.test(entry))
|
|
38
|
+
out.push(path);
|
|
39
|
+
}
|
|
40
|
+
};
|
|
41
|
+
walk(root);
|
|
42
|
+
return out;
|
|
43
|
+
}
|
|
44
|
+
/**
|
|
45
|
+
* Files a test could plausibly cover. Documentation, lockfiles and images change in almost every
|
|
46
|
+
* pull request; saying "nothing tests ROADMAP.md" is noise, and noise is how a review gets skipped.
|
|
47
|
+
*/
|
|
48
|
+
export const isCode = (file) => /\.(?:[cm]?[jt]sx?|py|rb|go|java|kt|cs|php|rs|vue|svelte|scala|swift|sql)$/i.test(file) && !/(?:^|[\/])(?:package-lock\.json|yarn\.lock)$/i.test(file);
|
|
49
|
+
/** The module names a file can be imported by: its path, its name, and its folder's front door. */
|
|
50
|
+
function importNames(root, changed) {
|
|
51
|
+
const full = resolve(root, changed);
|
|
52
|
+
const noExt = full.slice(0, full.length - extname(full).length);
|
|
53
|
+
const names = new Set([basename(noExt), relative(root, noExt).split(sep).join('/')]);
|
|
54
|
+
// Python: a.b.c for src/a/b/c.py, and the package it sits in.
|
|
55
|
+
const parts = relative(root, noExt).split(sep).filter((p) => p !== 'src');
|
|
56
|
+
names.add(parts.join('.'));
|
|
57
|
+
names.add(parts.slice(0, -1).join('.'));
|
|
58
|
+
if (basename(noExt) === 'index' || basename(noExt) === '__init__')
|
|
59
|
+
names.add(basename(dirname(noExt)));
|
|
60
|
+
return [...names].filter((n) => n.length > 1);
|
|
61
|
+
}
|
|
62
|
+
/**
|
|
63
|
+
* The package a changed file is re-exported from. In a project with one front door - which is most
|
|
64
|
+
* libraries - every test imports "../src" and nothing imports "../src/utils.ts", so without this a
|
|
65
|
+
* review finds no tests at all for a real change. Measured: 8 of 16 real commits were missed this
|
|
66
|
+
* way before.
|
|
67
|
+
*/
|
|
68
|
+
function packageNames(root, changed) {
|
|
69
|
+
const full = resolve(root, changed);
|
|
70
|
+
const out = [];
|
|
71
|
+
let dir = dirname(full);
|
|
72
|
+
for (let up = 0; up < 3; up++) {
|
|
73
|
+
for (const entry of ['index.ts', 'index.js', 'index.mjs', 'index.tsx', '__init__.py']) {
|
|
74
|
+
const front = join(dir, entry);
|
|
75
|
+
if (!existsSync(front))
|
|
76
|
+
continue;
|
|
77
|
+
let source = '';
|
|
78
|
+
try {
|
|
79
|
+
source = readFileSync(front, 'utf8');
|
|
80
|
+
}
|
|
81
|
+
catch {
|
|
82
|
+
continue;
|
|
83
|
+
}
|
|
84
|
+
const stem = basename(full).replace(/\.[^.]+$/, '');
|
|
85
|
+
// Either it names this file, or it re-exports everything beside it.
|
|
86
|
+
if (front !== full && (source.includes(stem) || /export\s+\*/.test(source))) {
|
|
87
|
+
out.push(relative(root, dir).split(sep).join('/'), basename(dir));
|
|
88
|
+
}
|
|
89
|
+
}
|
|
90
|
+
const parent = dirname(dir);
|
|
91
|
+
if (parent === dir)
|
|
92
|
+
break;
|
|
93
|
+
dir = parent;
|
|
94
|
+
}
|
|
95
|
+
return [...new Set(out)].filter(Boolean);
|
|
96
|
+
}
|
|
97
|
+
/** Does this test's source import that changed file? */
|
|
98
|
+
function importsIt(source, names) {
|
|
99
|
+
for (const name of names) {
|
|
100
|
+
const quoted = name.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
101
|
+
const patterns = [
|
|
102
|
+
new RegExp(`(?:from|import|require\\()\\s*['"][^'"]*${quoted}(?:\\.[cm]?[jt]sx?)?['"]`),
|
|
103
|
+
new RegExp(`(?:^|\\n)\\s*(?:from\\s+${quoted}\\s+import|import\\s+${quoted})\\b`),
|
|
104
|
+
];
|
|
105
|
+
if (patterns.some((p) => p.test(source)))
|
|
106
|
+
return true;
|
|
107
|
+
}
|
|
108
|
+
return false;
|
|
109
|
+
}
|
|
110
|
+
const words = (text) => (text.toLowerCase().match(/[a-z][a-z0-9]{2,}/g) ?? []).filter((w) => !STOP.has(w));
|
|
111
|
+
const STOP = new Set(['src', 'lib', 'app', 'test', 'tests', 'spec', 'index', 'main', 'the', 'and', 'for', 'com', 'org', 'utils', 'util', 'js', 'ts', 'py']);
|
|
112
|
+
/**
|
|
113
|
+
* The tests that reach a set of changed files. Imports first; then, for changed files nothing
|
|
114
|
+
* imports, the tests whose words overlap - marked as a guess.
|
|
115
|
+
*/
|
|
116
|
+
export function coverageFor(root, changedAll, tests = testFiles(root)) {
|
|
117
|
+
const changed = changedAll.filter(isCode);
|
|
118
|
+
const sources = new Map();
|
|
119
|
+
const read = (file) => {
|
|
120
|
+
if (!sources.has(file)) {
|
|
121
|
+
try {
|
|
122
|
+
sources.set(file, readFileSync(file, 'utf8'));
|
|
123
|
+
}
|
|
124
|
+
catch {
|
|
125
|
+
sources.set(file, '');
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
return sources.get(file);
|
|
129
|
+
};
|
|
130
|
+
const byTest = new Map();
|
|
131
|
+
const coveredByImport = new Set();
|
|
132
|
+
const add = (test, file, how) => {
|
|
133
|
+
const rel = relative(root, test).split(sep).join('/');
|
|
134
|
+
const found = byTest.get(rel) ?? { testFile: rel, covers: [], how };
|
|
135
|
+
// A test that imports the file itself is never downgraded to the weaker kinds of knowing.
|
|
136
|
+
if (RANK[how] < RANK[found.how])
|
|
137
|
+
found.how = how;
|
|
138
|
+
if (!found.covers.includes(file))
|
|
139
|
+
found.covers.push(file);
|
|
140
|
+
byTest.set(rel, found);
|
|
141
|
+
};
|
|
142
|
+
for (const file of changed) {
|
|
143
|
+
if (!existsSync(resolve(root, file)))
|
|
144
|
+
continue;
|
|
145
|
+
const direct = importNames(root, file);
|
|
146
|
+
const viaPackage = packageNames(root, file);
|
|
147
|
+
for (const test of tests) {
|
|
148
|
+
const source = read(test);
|
|
149
|
+
if (importsIt(source, direct)) {
|
|
150
|
+
coveredByImport.add(file);
|
|
151
|
+
add(test, file, 'imports');
|
|
152
|
+
}
|
|
153
|
+
else if (viaPackage.length > 0 && importsIt(source, viaPackage)) {
|
|
154
|
+
coveredByImport.add(file);
|
|
155
|
+
add(test, file, 'package');
|
|
156
|
+
}
|
|
157
|
+
}
|
|
158
|
+
}
|
|
159
|
+
// Only for the changed files nothing imports: the words, which is a weaker kind of knowing.
|
|
160
|
+
const orphans = changed.filter((f) => !coveredByImport.has(f));
|
|
161
|
+
for (const file of orphans) {
|
|
162
|
+
const wanted = new Set(words(file));
|
|
163
|
+
if (wanted.size === 0)
|
|
164
|
+
continue;
|
|
165
|
+
for (const test of tests) {
|
|
166
|
+
const rel = relative(root, test).split(sep).join('/');
|
|
167
|
+
if (byTest.has(rel))
|
|
168
|
+
continue;
|
|
169
|
+
const shared = words(`${rel} ${read(test).slice(0, 4000)}`).filter((w) => wanted.has(w));
|
|
170
|
+
if (new Set(shared).size < 2)
|
|
171
|
+
continue;
|
|
172
|
+
add(test, file, 'name');
|
|
173
|
+
}
|
|
174
|
+
}
|
|
175
|
+
// Direct imports first, then the package ones - and inside each group the test whose words match
|
|
176
|
+
// the change, because "every test in the library covers this" is true and useless.
|
|
177
|
+
const overlap = (test) => {
|
|
178
|
+
const wanted = new Set(test.covers.flatMap((f) => words(f)));
|
|
179
|
+
return words(test.testFile).filter((w) => wanted.has(w)).length;
|
|
180
|
+
};
|
|
181
|
+
return [...byTest.values()].sort((a, b) => RANK[a.how] - RANK[b.how] || overlap(b) - overlap(a) || b.covers.length - a.covers.length);
|
|
182
|
+
}
|
|
183
|
+
const RANK = { imports: 0, package: 1, name: 2 };
|
|
184
|
+
/** Changed files that no test reaches at all: the part of a change nobody will notice breaking. */
|
|
185
|
+
export function uncovered(changed, coverage) {
|
|
186
|
+
const reached = new Set(coverage.flatMap((c) => c.covers));
|
|
187
|
+
return changed.filter((f) => isCode(f) && !reached.has(f) && !TEST_FILE.test(f));
|
|
188
|
+
}
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
// `tewip review --diff main...HEAD`: what a developer wants to know before anyone reads the diff.
|
|
2
|
+
import { existsSync, readdirSync, writeFileSync } from 'node:fs';
|
|
3
|
+
import { join, resolve } from 'node:path';
|
|
4
|
+
import { changeSummary } from '../quality/grounding.js';
|
|
5
|
+
import { chooseFramework, detectFrameworks } from '../quality/frameworks.js';
|
|
6
|
+
import { readTriageRecords, datasetFiles } from '../../reporters/store.js';
|
|
7
|
+
import { coverageFor, uncovered } from './coverage.js';
|
|
8
|
+
import { renderReview, reviewChange } from './review.js';
|
|
9
|
+
/** Every failure Tewip has diagnosed in this project, from the history it keeps. */
|
|
10
|
+
function history(root, only) {
|
|
11
|
+
const dir = join(root, '.tewip', 'history');
|
|
12
|
+
if (!existsSync(dir))
|
|
13
|
+
return [];
|
|
14
|
+
const names = only ? [only] : readdirSync(dir);
|
|
15
|
+
return names.flatMap((name) => readTriageRecords(datasetFiles(join(dir, name)).triage));
|
|
16
|
+
}
|
|
17
|
+
export async function runReview(cmd) {
|
|
18
|
+
const change = changeSummary(cmd.root, cmd.diff);
|
|
19
|
+
if (!change)
|
|
20
|
+
throw new Error(`could not read the change "${cmd.diff}". Is this a git checkout, and does that range exist? (try --diff main...HEAD)`);
|
|
21
|
+
if (change.files.length === 0) {
|
|
22
|
+
console.log(`Nothing changed in ${cmd.diff}.`);
|
|
23
|
+
return null;
|
|
24
|
+
}
|
|
25
|
+
const files = change.files.map((f) => f.split('\\').join('/'));
|
|
26
|
+
const coverage = coverageFor(cmd.root, files);
|
|
27
|
+
const past = history(cmd.root, cmd.name);
|
|
28
|
+
const review = reviewChange(cmd.diff, files, coverage, past, uncovered(files, coverage));
|
|
29
|
+
// The command that runs just those tests - but only when the project's runner really owns them.
|
|
30
|
+
// Printing a Playwright command for a file Playwright never runs is worse than printing nothing.
|
|
31
|
+
const framework = chooseFramework(detectFrameworks(cmd.root), undefined, false) ?? detectFrameworks(cmd.root)[0] ?? null;
|
|
32
|
+
const ownsThem = framework ? review.runFirst.some((f) => resolve(cmd.root, f).startsWith(resolve(framework.testDir))) : false;
|
|
33
|
+
const runCommand = framework && ownsThem ? (chosen) => chosen.map((f) => framework.command(resolve(cmd.root, f))).join(' && ') : undefined;
|
|
34
|
+
const report = renderReview(review, runCommand);
|
|
35
|
+
console.log(report);
|
|
36
|
+
if (past.length === 0) {
|
|
37
|
+
console.log('\nNo failure history yet, so the "what has gone wrong here before" part is empty. It fills itself: run `tewip explain` or `tewip report` on your CI reports and every later review knows more.');
|
|
38
|
+
}
|
|
39
|
+
if (cmd.out) {
|
|
40
|
+
writeFileSync(resolve(cmd.out), `${report}\n`, 'utf8');
|
|
41
|
+
console.log(`\nWritten to ${cmd.out}`);
|
|
42
|
+
}
|
|
43
|
+
return review;
|
|
44
|
+
}
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
// The risk review of a change, from what Tewip already knows about this project.
|
|
2
|
+
//
|
|
3
|
+
// A general coding agent can read a diff. What it cannot do is tell you that the two tests covering
|
|
4
|
+
// your change have flaked eleven times this month, that this area has had three real bugs since
|
|
5
|
+
// August, and that nothing at all covers the file you edited. That comes from Tewip's own history,
|
|
6
|
+
// which is why this costs nothing and why nothing else can produce it.
|
|
7
|
+
import { componentOf } from '../../outlets/items.js';
|
|
8
|
+
const fileOf = (record) => record.file ?? '';
|
|
9
|
+
const countBy = (xs, key) => {
|
|
10
|
+
const out = new Map();
|
|
11
|
+
for (const x of xs)
|
|
12
|
+
out.set(key(x), (out.get(key(x)) ?? 0) + 1);
|
|
13
|
+
return out;
|
|
14
|
+
};
|
|
15
|
+
/** The review, assembled from the change, what covers it, and what has gone wrong here before. */
|
|
16
|
+
export function reviewChange(range, changed, coverage, history, notCovered) {
|
|
17
|
+
const byFile = new Map();
|
|
18
|
+
for (const record of history) {
|
|
19
|
+
const file = fileOf(record).split('\\').join('/');
|
|
20
|
+
if (!file)
|
|
21
|
+
continue;
|
|
22
|
+
byFile.set(file, [...(byFile.get(file) ?? []), record]);
|
|
23
|
+
}
|
|
24
|
+
const tests = coverage.map((c) => {
|
|
25
|
+
const past = byFile.get(c.testFile) ?? [];
|
|
26
|
+
const kinds = countBy(past, (r) => r.diagnosis?.category ?? 'UNKNOWN');
|
|
27
|
+
return {
|
|
28
|
+
testFile: c.testFile,
|
|
29
|
+
how: c.how,
|
|
30
|
+
covers: c.covers,
|
|
31
|
+
flakedTimes: kinds.get('TIMING') ?? 0,
|
|
32
|
+
bugTimes: kinds.get('REAL_BUG') ?? 0,
|
|
33
|
+
lastFailed: past.map((r) => r.timestamp).sort().at(-1) ?? '',
|
|
34
|
+
};
|
|
35
|
+
});
|
|
36
|
+
// The areas a change touches, and what has broken in them before.
|
|
37
|
+
const areaOf = (file) => componentOf(file);
|
|
38
|
+
const areas = new Map();
|
|
39
|
+
for (const file of changed) {
|
|
40
|
+
const area = areaOf(file);
|
|
41
|
+
const found = areas.get(area) ?? { area, failures: 0, realBugs: 0, flaky: 0, files: [] };
|
|
42
|
+
found.files.push(file);
|
|
43
|
+
areas.set(area, found);
|
|
44
|
+
}
|
|
45
|
+
for (const record of history) {
|
|
46
|
+
const area = areaOf(fileOf(record));
|
|
47
|
+
const found = areas.get(area);
|
|
48
|
+
if (!found)
|
|
49
|
+
continue;
|
|
50
|
+
found.failures++;
|
|
51
|
+
if (record.diagnosis?.category === 'REAL_BUG')
|
|
52
|
+
found.realBugs++;
|
|
53
|
+
if (record.diagnosis?.category === 'TIMING')
|
|
54
|
+
found.flaky++;
|
|
55
|
+
}
|
|
56
|
+
// What to run first: what really imports the change, worst history first - a test that has caught
|
|
57
|
+
// a real bug here before is the one most worth running now.
|
|
58
|
+
const runFirst = [...tests]
|
|
59
|
+
.filter((t) => t.how !== 'name')
|
|
60
|
+
.sort((a, b) => b.bugTimes - a.bugTimes || a.flakedTimes - b.flakedTimes)
|
|
61
|
+
.slice(0, 10)
|
|
62
|
+
.map((t) => t.testFile);
|
|
63
|
+
const notes = [];
|
|
64
|
+
const flaky = tests.filter((t) => t.flakedTimes >= 3);
|
|
65
|
+
if (flaky.length > 0) {
|
|
66
|
+
notes.push(`${flaky.length} of the tests covering this change have flaked before (${flaky.map((t) => `${t.testFile.split('/').at(-1)} ×${t.flakedTimes}`).join(', ')}). If one of them goes red, check Tewip's diagnosis before blaming the change.`);
|
|
67
|
+
}
|
|
68
|
+
if (notCovered.length > 0) {
|
|
69
|
+
notes.push(`${notCovered.length} changed file(s) have no test that imports them: ${notCovered.slice(0, 5).join(', ')}${notCovered.length > 5 ? ', ...' : ''}. A mistake there would reach production unnoticed.`);
|
|
70
|
+
}
|
|
71
|
+
const risky = [...areas.values()].filter((a) => a.realBugs >= 2).sort((a, b) => b.realBugs - a.realBugs);
|
|
72
|
+
if (risky.length > 0) {
|
|
73
|
+
notes.push(`${risky[0].area} has had ${risky[0].realBugs} real bug(s) in the history Tewip has kept. It is the part of this change worth a second pair of eyes.`);
|
|
74
|
+
}
|
|
75
|
+
if (tests.length === 0 && changed.length > 0) {
|
|
76
|
+
notes.push('No test in this project reaches any of the changed files. Tewip can write one: `tewip plan "<what the change should do>"` then `tewip write`.');
|
|
77
|
+
}
|
|
78
|
+
return { range, files: changed, tests, uncovered: notCovered, areas: [...areas.values()].sort((a, b) => b.realBugs - a.realBugs || b.failures - a.failures), runFirst, notes };
|
|
79
|
+
}
|
|
80
|
+
const when = (at) => {
|
|
81
|
+
if (!at)
|
|
82
|
+
return 'never';
|
|
83
|
+
const days = Math.round((Date.now() - Date.parse(at)) / 86_400_000);
|
|
84
|
+
if (!Number.isFinite(days))
|
|
85
|
+
return 'once';
|
|
86
|
+
return days <= 1 ? 'yesterday' : days < 14 ? `${days} days ago` : days < 60 ? `${Math.round(days / 7)} weeks ago` : `${Math.round(days / 30)} months ago`;
|
|
87
|
+
};
|
|
88
|
+
/** The review as a person reads it, before they read the diff. */
|
|
89
|
+
export function renderReview(review, runCommand) {
|
|
90
|
+
const lines = [`# Risk review: ${review.range}`, '', `${review.files.length} changed file(s).`, ''];
|
|
91
|
+
if (review.notes.length > 0) {
|
|
92
|
+
lines.push('## Before you read the diff', '');
|
|
93
|
+
for (const note of review.notes)
|
|
94
|
+
lines.push(`- ${note}`);
|
|
95
|
+
lines.push('');
|
|
96
|
+
}
|
|
97
|
+
lines.push('## Tests that cover this change', '');
|
|
98
|
+
if (review.tests.length === 0)
|
|
99
|
+
lines.push('None found.', '');
|
|
100
|
+
for (const test of review.tests.slice(0, 15)) {
|
|
101
|
+
const past = test.flakedTimes + test.bugTimes > 0 ? ` — failed before: ${test.bugTimes} real bug(s), ${test.flakedTimes} flaky, last ${when(test.lastFailed)}` : ' — never failed in the history Tewip has kept';
|
|
102
|
+
const how = test.how === 'imports' ? 'imports the changed code' : test.how === 'package' ? 'imports the package that exports it' : 'shares its words, so this one is a guess';
|
|
103
|
+
lines.push(`- **${test.testFile}** (${how})${past}`);
|
|
104
|
+
lines.push(` - covers: ${test.covers.slice(0, 4).join(', ')}${test.covers.length > 4 ? `, +${test.covers.length - 4}` : ''}`);
|
|
105
|
+
}
|
|
106
|
+
lines.push('');
|
|
107
|
+
if (review.uncovered.length > 0) {
|
|
108
|
+
lines.push('## Changed, but nothing tests it', '');
|
|
109
|
+
for (const file of review.uncovered.slice(0, 15))
|
|
110
|
+
lines.push(`- ${file}`);
|
|
111
|
+
lines.push('');
|
|
112
|
+
}
|
|
113
|
+
if (review.areas.some((a) => a.failures > 0)) {
|
|
114
|
+
lines.push('## What has gone wrong here before', '');
|
|
115
|
+
for (const area of review.areas.filter((a) => a.failures > 0).slice(0, 6)) {
|
|
116
|
+
lines.push(`- **${area.area}**: ${area.failures} failure(s) — ${area.realBugs} real bug(s), ${area.flaky} flaky`);
|
|
117
|
+
}
|
|
118
|
+
lines.push('');
|
|
119
|
+
}
|
|
120
|
+
if (review.runFirst.length > 0) {
|
|
121
|
+
lines.push('## Run these first', '');
|
|
122
|
+
lines.push(runCommand ? '```' : '', runCommand ? runCommand(review.runFirst) : review.runFirst.map((f) => `- ${f}`).join('\n'), runCommand ? '```' : '');
|
|
123
|
+
lines.push('');
|
|
124
|
+
}
|
|
125
|
+
lines.push('---', '', 'Read from this project\'s own failure history. No model was called, so this cost nothing.');
|
|
126
|
+
return lines.join('\n');
|
|
127
|
+
}
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
export function azureFromEnv(env = process.env) {
|
|
2
|
+
const org = env.TEWIP_AZURE_ORG?.trim();
|
|
3
|
+
const project = env.TEWIP_AZURE_PROJECT?.trim();
|
|
4
|
+
if (!org || !project)
|
|
5
|
+
return null;
|
|
6
|
+
return { org, project, token: (env.TEWIP_AZURE_TOKEN ?? env.SYSTEM_ACCESSTOKEN ?? '').trim() };
|
|
7
|
+
}
|
|
8
|
+
/** Azure's own names for the kinds of work. A project template may not have all of them. */
|
|
9
|
+
const AZURE_TYPE = {
|
|
10
|
+
'test-case': ['Test Case', 'Task'],
|
|
11
|
+
bug: ['Bug', 'Issue'],
|
|
12
|
+
story: ['User Story', 'Product Backlog Item', 'Issue'],
|
|
13
|
+
epic: ['Epic', 'Feature'],
|
|
14
|
+
};
|
|
15
|
+
export class AzureBoards {
|
|
16
|
+
name = 'Azure Boards';
|
|
17
|
+
config;
|
|
18
|
+
constructor(config) {
|
|
19
|
+
this.config = config;
|
|
20
|
+
}
|
|
21
|
+
canWrite() {
|
|
22
|
+
return Boolean(this.config.token);
|
|
23
|
+
}
|
|
24
|
+
headers(contentType = 'application/json') {
|
|
25
|
+
// Azure takes the token as the password of a basic login with an empty user.
|
|
26
|
+
return { Authorization: `Basic ${Buffer.from(`:${this.config.token}`).toString('base64')}`, Accept: 'application/json', 'Content-Type': contentType };
|
|
27
|
+
}
|
|
28
|
+
base() {
|
|
29
|
+
return `https://dev.azure.com/${encodeURIComponent(this.config.org)}/${encodeURIComponent(this.config.project)}/_apis`;
|
|
30
|
+
}
|
|
31
|
+
async request(method, url, body, contentType) {
|
|
32
|
+
const res = await fetch(url, { method, headers: this.headers(contentType), body: body ? JSON.stringify(body) : undefined });
|
|
33
|
+
if (!res.ok)
|
|
34
|
+
throw new Error(`Azure Boards ${method} ${url.replace(/\?.*/, '')} -> ${res.status}: ${(await res.text()).slice(0, 300)}`);
|
|
35
|
+
return (res.status === 204 ? undefined : await res.json());
|
|
36
|
+
}
|
|
37
|
+
/** Work items carrying Tewip's tag. Azure answers a query with ids, which are then read. */
|
|
38
|
+
async mine(_project, label) {
|
|
39
|
+
const wiql = { query: `SELECT [System.Id] FROM WorkItems WHERE [System.Tags] CONTAINS '${label.replace(/'/g, "''")}'` };
|
|
40
|
+
const found = await this.request('POST', `${this.base()}/wit/wiql?api-version=7.0`, wiql);
|
|
41
|
+
const ids = (found.workItems ?? []).slice(0, 200).map((w) => w.id);
|
|
42
|
+
if (ids.length === 0)
|
|
43
|
+
return [];
|
|
44
|
+
const items = await this.request('GET', `${this.base()}/wit/workitems?ids=${ids.join(',')}&fields=System.Title&api-version=7.0`);
|
|
45
|
+
return (items.value ?? []).map((i) => ({ key: String(i.id), title: i.fields['System.Title'] ?? '' }));
|
|
46
|
+
}
|
|
47
|
+
async create(_project, item) {
|
|
48
|
+
// Azure creates a work item from a list of patch operations, not from an object.
|
|
49
|
+
const type = AZURE_TYPE[item.kind][0];
|
|
50
|
+
const patch = [
|
|
51
|
+
{ op: 'add', path: '/fields/System.Title', value: item.title.slice(0, 250) },
|
|
52
|
+
{ op: 'add', path: '/fields/System.Description', value: item.body.slice(0, 60_000).replace(/\n/g, '<br>') },
|
|
53
|
+
];
|
|
54
|
+
if (item.labels?.length)
|
|
55
|
+
patch.push({ op: 'add', path: '/fields/System.Tags', value: item.labels.join('; ') });
|
|
56
|
+
const created = await this.request('POST', `${this.base()}/wit/workitems/$${encodeURIComponent(type)}?api-version=7.0`, patch, 'application/json-patch+json');
|
|
57
|
+
return String(created.id);
|
|
58
|
+
}
|
|
59
|
+
async comment(key, text) {
|
|
60
|
+
await this.request('POST', `${this.base()}/wit/workItems/${encodeURIComponent(key)}/comments?api-version=7.0-preview.3`, { text: text.slice(0, 60_000) });
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
export function linearFromEnv(env = process.env) {
|
|
64
|
+
const apiKey = env.TEWIP_LINEAR_KEY?.trim();
|
|
65
|
+
return apiKey ? { apiKey, team: (env.TEWIP_LINEAR_TEAM ?? '').trim() } : null;
|
|
66
|
+
}
|
|
67
|
+
export class Linear {
|
|
68
|
+
name = 'Linear';
|
|
69
|
+
config;
|
|
70
|
+
teamId = null;
|
|
71
|
+
constructor(config) {
|
|
72
|
+
this.config = config;
|
|
73
|
+
}
|
|
74
|
+
canWrite() {
|
|
75
|
+
return Boolean(this.config.apiKey && this.config.team);
|
|
76
|
+
}
|
|
77
|
+
/** Linear speaks GraphQL, and answers 200 with the errors inside the body. */
|
|
78
|
+
async graphql(query, variables = {}) {
|
|
79
|
+
const res = await fetch('https://api.linear.app/graphql', {
|
|
80
|
+
method: 'POST',
|
|
81
|
+
headers: { Authorization: this.config.apiKey, 'Content-Type': 'application/json' },
|
|
82
|
+
body: JSON.stringify({ query, variables }),
|
|
83
|
+
});
|
|
84
|
+
if (!res.ok)
|
|
85
|
+
throw new Error(`Linear -> ${res.status}: ${(await res.text()).slice(0, 300)}`);
|
|
86
|
+
const body = (await res.json());
|
|
87
|
+
if (body.errors?.length)
|
|
88
|
+
throw new Error(`Linear: ${body.errors.map((e) => e.message).join('; ').slice(0, 300)}`);
|
|
89
|
+
return body.data;
|
|
90
|
+
}
|
|
91
|
+
async team() {
|
|
92
|
+
if (this.teamId)
|
|
93
|
+
return this.teamId;
|
|
94
|
+
const data = await this.graphql('query { teams(first: 50) { nodes { id key } } }');
|
|
95
|
+
const found = data.teams.nodes.find((t) => t.key.toLowerCase() === this.config.team.toLowerCase());
|
|
96
|
+
if (!found)
|
|
97
|
+
throw new Error(`Linear has no team with the key "${this.config.team}". It has: ${data.teams.nodes.map((t) => t.key).join(', ')}`);
|
|
98
|
+
this.teamId = found.id;
|
|
99
|
+
return found.id;
|
|
100
|
+
}
|
|
101
|
+
async mine(_project, label) {
|
|
102
|
+
const data = await this.graphql('query($label: String!) { issues(first: 100, filter: { labels: { name: { eq: $label } } }) { nodes { identifier title url } } }', { label });
|
|
103
|
+
return data.issues.nodes.map((i) => ({ key: i.identifier, title: i.title, url: i.url }));
|
|
104
|
+
}
|
|
105
|
+
async create(_project, item) {
|
|
106
|
+
const teamId = await this.team();
|
|
107
|
+
const data = await this.graphql('mutation($input: IssueCreateInput!) { issueCreate(input: $input) { issue { identifier } } }', { input: { teamId, title: item.title.slice(0, 250), description: item.body.slice(0, 60_000) } });
|
|
108
|
+
return data.issueCreate.issue.identifier;
|
|
109
|
+
}
|
|
110
|
+
async comment(key, text) {
|
|
111
|
+
const found = await this.graphql('query($id: String!) { issue(id: $id) { id } }', { id: key });
|
|
112
|
+
if (!found.issue)
|
|
113
|
+
throw new Error(`Linear has no issue ${key}`);
|
|
114
|
+
await this.graphql('mutation($input: CommentCreateInput!) { commentCreate(input: $input) { success } }', { input: { issueId: found.issue.id, body: text.slice(0, 60_000) } });
|
|
115
|
+
}
|
|
116
|
+
}
|
|
@@ -4,17 +4,22 @@ export const PRIORITY_WORDS = {
|
|
|
4
4
|
jira: { P0: 'Highest', P1: 'High', P2: 'Medium' },
|
|
5
5
|
rally: { P0: 'Resolve Immediately', P1: 'High Attention', P2: 'Normal' },
|
|
6
6
|
github: { P0: 'priority:high', P1: 'priority:medium', P2: 'priority:low' },
|
|
7
|
+
azure: { P0: '1', P1: '2', P2: '3' },
|
|
8
|
+
linear: { P0: 'Urgent', P1: 'High', P2: 'Medium' },
|
|
7
9
|
};
|
|
8
10
|
export function priorityWord(tracker, priority) {
|
|
9
11
|
return PRIORITY_WORDS[tracker]?.[priority] ?? priority;
|
|
10
12
|
}
|
|
11
|
-
/** Which tracker the settings point at. Jira stays the default, because most teams are there. */
|
|
12
13
|
export function trackerNameFromEnv(env = process.env) {
|
|
13
14
|
const named = (env.TEWIP_TRACKER ?? '').trim().toLowerCase();
|
|
14
|
-
if (
|
|
15
|
+
if (['jira', 'rally', 'github', 'azure', 'linear'].includes(named))
|
|
15
16
|
return named;
|
|
16
17
|
if (env.TEWIP_RALLY_KEY)
|
|
17
18
|
return 'rally';
|
|
19
|
+
if (env.TEWIP_LINEAR_KEY)
|
|
20
|
+
return 'linear';
|
|
21
|
+
if (env.TEWIP_AZURE_ORG && env.TEWIP_AZURE_PROJECT)
|
|
22
|
+
return 'azure';
|
|
18
23
|
if (env.TEWIP_GITHUB_REPO && !env.TEWIP_JIRA_URL)
|
|
19
24
|
return 'github';
|
|
20
25
|
return 'jira';
|
|
@@ -32,6 +37,12 @@ export function trackerHelp() {
|
|
|
32
37
|
' GitHub Issues',
|
|
33
38
|
' TEWIP_GITHUB_REPO=owner/name TEWIP_GITHUB_TOKEN=ghp_... (or be logged in with the gh CLI)',
|
|
34
39
|
'',
|
|
35
|
-
'
|
|
40
|
+
' Azure DevOps Boards',
|
|
41
|
+
' TEWIP_AZURE_ORG=your-org TEWIP_AZURE_PROJECT=your-project TEWIP_AZURE_TOKEN=...',
|
|
42
|
+
'',
|
|
43
|
+
' Linear',
|
|
44
|
+
' TEWIP_LINEAR_KEY=lin_api_... TEWIP_LINEAR_TEAM=ENG',
|
|
45
|
+
'',
|
|
46
|
+
'Set TEWIP_TRACKER=jira|rally|github|azure|linear when more than one is configured.',
|
|
36
47
|
].join('\n');
|
|
37
48
|
}
|
|
@@ -6,6 +6,7 @@
|
|
|
6
6
|
import { Jira, jiraFromEnv } from './jira.js';
|
|
7
7
|
import { Rally, rallyFromEnv } from './rally.js';
|
|
8
8
|
import { GitHubIssues, githubIssuesFromEnv } from './github-issues.js';
|
|
9
|
+
import { AzureBoards, azureFromEnv, Linear, linearFromEnv } from './azure-linear.js';
|
|
9
10
|
import { priorityWord, trackerHelp, trackerNameFromEnv } from './tracker.js';
|
|
10
11
|
/** Jira, wearing the same interface as the rest. */
|
|
11
12
|
export class JiraTracker {
|
|
@@ -66,6 +67,14 @@ export function trackerFromEnv(projectKey = '', env = process.env) {
|
|
|
66
67
|
const config = githubIssuesFromEnv(env);
|
|
67
68
|
return config ? { tracker: new GitHubIssues(config), project: projectKey || config.repo, id } : null;
|
|
68
69
|
}
|
|
70
|
+
if (id === 'azure') {
|
|
71
|
+
const config = azureFromEnv(env);
|
|
72
|
+
return config ? { tracker: new AzureBoards(config), project: projectKey || config.project, id } : null;
|
|
73
|
+
}
|
|
74
|
+
if (id === 'linear') {
|
|
75
|
+
const config = linearFromEnv(env);
|
|
76
|
+
return config ? { tracker: new Linear(config), project: projectKey || config.team, id } : null;
|
|
77
|
+
}
|
|
69
78
|
const config = jiraFromEnv();
|
|
70
79
|
return config ? { tracker: new JiraTracker(new Jira(config), Boolean(config.token)), project: projectKey, id } : null;
|
|
71
80
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tewip-ai",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"test-failures",
|