@regressionbot/sdk 1.2.2 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +186 -4
- package/dist/cli.js +29 -13
- package/dist/index.d.ts +25 -8
- package/dist/index.js +47 -10
- package/dist/types.d.ts +237 -28
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -4,11 +4,16 @@ The official SDK for [RegressionBot.com](https://regressionbot.com) — the simp
|
|
|
4
4
|
|
|
5
5
|
RegressionBot is a declarative visual regression testing platform that helps you catch UI changes before they reach production. This SDK provides a fluent, chainable API to define your test scope, run visual tests, and manage baselines programmatically.
|
|
6
6
|
|
|
7
|
+
[](https://regressionbot.com/docs)
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
|
|
7
12
|
## Why RegressionBot?
|
|
8
13
|
|
|
9
14
|
Unlike traditional visual diffing libraries, RegressionBot is designed for modern, automated development loops and agentic pipelines:
|
|
10
15
|
|
|
11
|
-
- **Highly Accurate Regressions (
|
|
16
|
+
- **Highly Accurate Regressions (Less Noise)**: Leveraging advanced pixel-matching algorithms and element masking (using CSS selectors or the automatic `data-vr-mask` attribute), RegressionBot minimizes false positives caused by dynamic data, layout shifting, or third-party widgets.
|
|
12
17
|
- **Plain-English Summaries**: No more manual screenshot comparisons. RegressionBot translates visual diffs into concise, plain-English descriptions of what changed, so you know exactly what was modified at a glance.
|
|
13
18
|
- **Agentic Workflow Ready**: Built from the ground up to support autonomous coding agents (like Gemini or Claude) and automated developer loops. Through standard API endpoints, CLI commands, and Model Context Protocol (MCP) integrations, agents can trigger tests, read plain-English results, and approve baseline changes programmatically without human intervention.
|
|
14
19
|
|
|
@@ -19,6 +24,8 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
|
|
|
19
24
|
- **Matrix Testing**: Test multiple devices and viewports in a single job.
|
|
20
25
|
- **Auto-Discovery**: Scan sitemaps with glob patterns and limits.
|
|
21
26
|
- **RegressionBot Summaries**: Plain-English change descriptions for every regression, generated on-demand via the API.
|
|
27
|
+
- **Intent-Aware Verdicts**: Describe what a run is meant to change, and every regression comes back judged as intentional, a bug, or noise.
|
|
28
|
+
- **Scheduled Checks**: Run a saved project unattended, hourly, daily, or weekly — and at a set UTC hour.
|
|
22
29
|
- **Project-Based Baselines**: Save and reuse test configurations; share visual history across environments.
|
|
23
30
|
- **Auto-Approval**: Automatically promote screenshots to baselines on jobs that pass your criteria.
|
|
24
31
|
- **Zero Infrastructure**: No browser maintenance or server provisioning — RegressionBot handles it all.
|
|
@@ -29,6 +36,44 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
|
|
|
29
36
|
npm install @regressionbot/sdk
|
|
30
37
|
```
|
|
31
38
|
|
|
39
|
+
## Migration from 1.x
|
|
40
|
+
|
|
41
|
+
2.0.0 re-types the SDK against what the API actually returns, taken from the
|
|
42
|
+
orchestrator's implementation rather than from a spec that had drifted in both
|
|
43
|
+
directions. Most of the breakage is caught by the compiler. One is not:
|
|
44
|
+
|
|
45
|
+
```typescript
|
|
46
|
+
// 1.x — updateProject() returned the raw { message, project } envelope,
|
|
47
|
+
// so every field read off it was undefined at runtime
|
|
48
|
+
const project = await rb.updateProject('site', { schedule: 'daily' });
|
|
49
|
+
project.project.schedule; // this is where the value actually was
|
|
50
|
+
|
|
51
|
+
// 2.0 — the envelope is unwrapped for you
|
|
52
|
+
const project = await rb.updateProject('site', { schedule: 'daily' });
|
|
53
|
+
project.schedule; // 'daily'
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
The rest surface as type errors:
|
|
57
|
+
|
|
58
|
+
- **`PageResult.diffCount` is gone.** No endpoint ever returned it — both
|
|
59
|
+
`getJobStatus` and `getJobSummary` strip it. `PageResult` gains the fields they
|
|
60
|
+
do return: `status`, `isNewBaseline`, `errorMessage`, `elementsChanged`,
|
|
61
|
+
`domAssistSkipReason`, `maskUrl` and `verdict`. Image URLs are typed nullable,
|
|
62
|
+
which they always were.
|
|
63
|
+
- **`updateProject()` takes `ProjectConfigUpdate`, not `Partial<ProjectConfig>`.**
|
|
64
|
+
The read and write shapes genuinely differ: a gate credential is written as an
|
|
65
|
+
`EnvGate` and read back only as `{ configured: true }`.
|
|
66
|
+
- **`ProjectConfig.orgId` is gone** — the API stopped returning it.
|
|
67
|
+
- **`summaryStatus` can be `FAILED`**, which the old union omitted. `error`,
|
|
68
|
+
`progress`, `results`, `createdAt` and `intentAssessment` are always returned.
|
|
69
|
+
- **`approve()` gains `conflictedUrls`**, returned when another job updated a
|
|
70
|
+
baseline first.
|
|
71
|
+
|
|
72
|
+
New in 2.0 and requiring a current API: `.customCss()` and `.withContext()` on
|
|
73
|
+
the builder, intent-aware verdicts, environment gates, baseline policies, and
|
|
74
|
+
scheduling. `scheduleHourUtc` needs API 2.7.0 or later — on an older API the
|
|
75
|
+
field is accepted and ignored, and the schedule stays anchored to its first run.
|
|
76
|
+
|
|
32
77
|
## Usage
|
|
33
78
|
|
|
34
79
|
### Basic Example
|
|
@@ -86,6 +131,12 @@ const job = await rb
|
|
|
86
131
|
.mask(['.ads', '#modal']) // Manual selectors
|
|
87
132
|
// Tip: Adding 'data-vr-mask' to your HTML elements masks them automatically!
|
|
88
133
|
|
|
134
|
+
// Custom CSS: Injected before every screenshot (max 4096 chars)
|
|
135
|
+
.customCss('#chat-widget { display: none !important; }')
|
|
136
|
+
|
|
137
|
+
// Intent: What this run is testing, so changes can be judged against it
|
|
138
|
+
.withContext({ changeDescription: 'Restyle the pricing table' })
|
|
139
|
+
|
|
89
140
|
// Execute: Compiles manifest and triggers the API
|
|
90
141
|
.run();
|
|
91
142
|
|
|
@@ -107,9 +158,9 @@ const summary = await job.getSummary();
|
|
|
107
158
|
if (summary.regressions.length > 0) {
|
|
108
159
|
console.log(`\n${summary.regressions.length} regressions found:`);
|
|
109
160
|
for (const regression of summary.regressions) {
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
console.log(
|
|
161
|
+
for (const item of regression.regressionbotSummary ?? []) {
|
|
162
|
+
// `label` is the region letter (A, B, C…), or '' for a whole-page change
|
|
163
|
+
console.log(`[${item.label}] ${item.text}`);
|
|
113
164
|
}
|
|
114
165
|
}
|
|
115
166
|
}
|
|
@@ -119,6 +170,68 @@ const aiResult = await job.generateAiSummary();
|
|
|
119
170
|
console.log(`Generated summaries for ${aiResult.summaries.length} regressions.`);
|
|
120
171
|
```
|
|
121
172
|
|
|
173
|
+
### Intent-Aware Verdicts
|
|
174
|
+
|
|
175
|
+
Tell RegressionBot what a run is meant to change, and every regression comes back judged
|
|
176
|
+
against that intent — `intentional`, `bug`, `noise`, or `needs_review`.
|
|
177
|
+
|
|
178
|
+
```typescript
|
|
179
|
+
const job = await rb
|
|
180
|
+
.test(process.env.VERCEL_PREVIEW_URL)
|
|
181
|
+
.forProject('marketing-site-v2')
|
|
182
|
+
.withContext({
|
|
183
|
+
changeDescription: 'Restyle the pricing table',
|
|
184
|
+
gitCommitSha: process.env.GITHUB_SHA,
|
|
185
|
+
prTitle: process.env.PR_TITLE,
|
|
186
|
+
expectedChanges: ['Pricing table background is now dark'],
|
|
187
|
+
scope: ['components/PricingTable.tsx'],
|
|
188
|
+
})
|
|
189
|
+
.run();
|
|
190
|
+
|
|
191
|
+
await job.waitForCompletion(2000, undefined, { waitForSummaries: true });
|
|
192
|
+
const summary = await job.getSummary();
|
|
193
|
+
|
|
194
|
+
// Whole-job roll-up: does everything that changed line up with the stated intent?
|
|
195
|
+
console.log(summary.intentAssessment?.summary);
|
|
196
|
+
console.log(`Bugs: ${summary.intentAssessment?.bugCount}`);
|
|
197
|
+
|
|
198
|
+
// Per-result verdict, plus the per-region verdicts behind it
|
|
199
|
+
for (const r of summary.regressions) {
|
|
200
|
+
console.log(`${r.url}: ${r.verdict?.decision} (${r.verdict?.avgConfidence})`);
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
// Fail CI only on changes that were not intended
|
|
204
|
+
const bugs = summary.regressions.filter(r => r.verdict?.decision === 'bug');
|
|
205
|
+
if (bugs.length > 0) process.exit(1);
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
`runProject()` takes the same context, so a run from a saved project can be
|
|
209
|
+
intent-aware too:
|
|
210
|
+
|
|
211
|
+
```typescript
|
|
212
|
+
const job = await rb.runProject('marketing-site-v2', {
|
|
213
|
+
testOrigin: process.env.VERCEL_PREVIEW_URL,
|
|
214
|
+
runContext: { changeDescription: 'Restyle the pricing table' },
|
|
215
|
+
});
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
### Hiding Dynamic Content
|
|
219
|
+
|
|
220
|
+
`.mask()` hides elements by selector. `.customCss()` injects arbitrary CSS before each
|
|
221
|
+
screenshot when masking isn't enough — collapsing an animation, pinning a carousel, or
|
|
222
|
+
neutralising a third-party widget. Max 4096 characters.
|
|
223
|
+
|
|
224
|
+
```typescript
|
|
225
|
+
const job = await rb
|
|
226
|
+
.test('https://preview.myapp.com')
|
|
227
|
+
.forProject('my-app-web')
|
|
228
|
+
.customCss(`
|
|
229
|
+
#chat-widget { display: none !important; }
|
|
230
|
+
* { animation: none !important; transition: none !important; }
|
|
231
|
+
`)
|
|
232
|
+
.run();
|
|
233
|
+
```
|
|
234
|
+
|
|
122
235
|
### Progress Tracking
|
|
123
236
|
|
|
124
237
|
Pass a callback to `waitForCompletion` to receive status updates while the job runs:
|
|
@@ -172,6 +285,69 @@ const summary = await job.getSummary();
|
|
|
172
285
|
console.log(`Score: ${summary.overallScore}/100`);
|
|
173
286
|
```
|
|
174
287
|
|
|
288
|
+
### Environment Gates
|
|
289
|
+
|
|
290
|
+
If an origin sits behind basic auth, a bypass header, or a session cookie, store the
|
|
291
|
+
credential on the project. It is encrypted at rest and never returned — reads report
|
|
292
|
+
only `{ configured: true }`.
|
|
293
|
+
|
|
294
|
+
```typescript
|
|
295
|
+
await rb.updateProject('marketing-site-v2', {
|
|
296
|
+
testAuth: { basic: { username: 'preview', password: process.env.PREVIEW_PASSWORD! } },
|
|
297
|
+
// or: { headers: { 'x-vercel-protection-bypass': token } }
|
|
298
|
+
// or: { cookies: [{ name: 'session', value: token }] }
|
|
299
|
+
});
|
|
300
|
+
|
|
301
|
+
const project = await rb.getProject('marketing-site-v2');
|
|
302
|
+
console.log(project.testAuth); // { configured: true }
|
|
303
|
+
|
|
304
|
+
// Clear it
|
|
305
|
+
await rb.updateProject('marketing-site-v2', { testAuth: null });
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
Setting or clearing a gate credential invalidates the stored baselines, since it changes
|
|
309
|
+
what the capture can reach.
|
|
310
|
+
|
|
311
|
+
### Scheduled Checks
|
|
312
|
+
|
|
313
|
+
A project can run unattended on a fixed cadence. Scheduling requires
|
|
314
|
+
`baselinePolicy: 'rolling'` on a managed project — on the default `approved` policy an
|
|
315
|
+
unapproved change would be re-reported on every subsequent run, so the API rejects the
|
|
316
|
+
combination. Live-vs-live projects store no baseline and are exempt.
|
|
317
|
+
|
|
318
|
+
```typescript
|
|
319
|
+
// Turn on a daily unattended check
|
|
320
|
+
await rb.updateProject('marketing-site-v2', {
|
|
321
|
+
baselinePolicy: 'rolling',
|
|
322
|
+
schedule: 'daily', // 'hourly' | 'daily' | 'weekly'
|
|
323
|
+
});
|
|
324
|
+
|
|
325
|
+
// Or pin it to a UTC hour — this one runs at 03:00 UTC every day
|
|
326
|
+
await rb.updateProject('marketing-site-v2', {
|
|
327
|
+
baselinePolicy: 'rolling',
|
|
328
|
+
schedule: 'daily',
|
|
329
|
+
scheduleHourUtc: 3, // 0–23, UTC
|
|
330
|
+
});
|
|
331
|
+
|
|
332
|
+
// Turn it back off (this clears scheduleHourUtc too)
|
|
333
|
+
await rb.updateProject('marketing-site-v2', { schedule: null });
|
|
334
|
+
|
|
335
|
+
// Read back when the scheduler last fired — the next run is due one interval after this
|
|
336
|
+
const project = await rb.getProject('marketing-site-v2');
|
|
337
|
+
console.log(project.schedule, project.scheduleHourUtc, project.lastScheduledRunAt);
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
Without `scheduleHourUtc` the first run starts at the next hourly sweep and the cadence
|
|
341
|
+
anchors to it. With it, the first run waits for that hour as well. Hours only, since the
|
|
342
|
+
scheduler sweeps once an hour, and UTC only. It is rejected on an `hourly` schedule, and
|
|
343
|
+
rejected on a project with no schedule to apply it to. Clearing the schedule clears the
|
|
344
|
+
hour with it, and a missed slot waits for the next one rather than catching up.
|
|
345
|
+
|
|
346
|
+
Setting a schedule, its hour, or a baseline policy does **not** invalidate
|
|
347
|
+
baselines — only changes that affect what a capture looks like (`testOrigin`,
|
|
348
|
+
`baseOrigin`, `sitemapUrl`, `paths`, `scans`, `devices`, `masks`, `customCss`) do that.
|
|
349
|
+
Billing is per comparison, so cost scales with frequency.
|
|
350
|
+
|
|
175
351
|
### Reconnecting to an Existing Job
|
|
176
352
|
|
|
177
353
|
If you have a job ID from a previous run (e.g., stored in CI state), you can attach to it without re-running the test:
|
|
@@ -205,6 +381,12 @@ Test an entire site using glob patterns.
|
|
|
205
381
|
npx @regressionbot/sdk https://example.com --project my-project --scan "/**" --exclude "/admin/**" --concurrency 20
|
|
206
382
|
```
|
|
207
383
|
|
|
384
|
+
Use `--mask` to hide elements by selector, or `--custom-css` to inject CSS before every
|
|
385
|
+
screenshot:
|
|
386
|
+
```bash
|
|
387
|
+
npx @regressionbot/sdk https://example.com --project my-site --custom-css "#chat-widget { display: none !important; }"
|
|
388
|
+
```
|
|
389
|
+
|
|
208
390
|
#### 3. Job Summary
|
|
209
391
|
Get detailed results and diff URLs for a completed job.
|
|
210
392
|
```bash
|
package/dist/cli.js
CHANGED
|
@@ -37,6 +37,23 @@ Object.defineProperty(exports, "__esModule", { value: true });
|
|
|
37
37
|
const index_1 = require("./index");
|
|
38
38
|
const security_1 = require("./security");
|
|
39
39
|
const path = __importStar(require("path"));
|
|
40
|
+
function formatSummary(items) {
|
|
41
|
+
if (!items || items.length === 0)
|
|
42
|
+
return '';
|
|
43
|
+
return items
|
|
44
|
+
.map(item => (item.label ? `[${item.label}] ${item.text}` : item.text))
|
|
45
|
+
.join('\n ');
|
|
46
|
+
}
|
|
47
|
+
function printRegression(r) {
|
|
48
|
+
console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
|
|
49
|
+
console.log(` Diff: ${r.diffUrl}`);
|
|
50
|
+
if (r.verdict) {
|
|
51
|
+
console.log(` Verdict: ${r.verdict.decision} (confidence ${r.verdict.minConfidence.toFixed(2)}-${r.verdict.avgConfidence.toFixed(2)})`);
|
|
52
|
+
}
|
|
53
|
+
const summary = formatSummary(r.regressionbotSummary);
|
|
54
|
+
if (summary)
|
|
55
|
+
console.log(` Summary: ${summary}`);
|
|
56
|
+
}
|
|
40
57
|
function parseArgs(args) {
|
|
41
58
|
const options = Object.create(null);
|
|
42
59
|
options._ = [];
|
|
@@ -129,6 +146,7 @@ Options for <url>:
|
|
|
129
146
|
--concurrency <n> Max concurrent workers (default 10).
|
|
130
147
|
--auto-approve Automatically approve results as new baselines.
|
|
131
148
|
--mask <selectors> Comma-separated CSS selectors to hide (e.g. ".ad,#popup").
|
|
149
|
+
--custom-css <css> CSS injected before each screenshot (max 4096 chars).
|
|
132
150
|
--skip-summaries Skip waiting for parallel AI summaries in the CLI.
|
|
133
151
|
|
|
134
152
|
Environment Variables:
|
|
@@ -167,6 +185,9 @@ async function startJob(url, options) {
|
|
|
167
185
|
const selectors = options.mask.split(',').map((s) => s.trim());
|
|
168
186
|
builder.mask(selectors);
|
|
169
187
|
}
|
|
188
|
+
if (typeof options['custom-css'] === 'string') {
|
|
189
|
+
builder.customCss(options['custom-css']);
|
|
190
|
+
}
|
|
170
191
|
const job = await builder.run();
|
|
171
192
|
console.log(`✅ Job started! ID: ${job.jobId}`);
|
|
172
193
|
console.log(`📊 Project: ${projectId}`);
|
|
@@ -179,7 +200,7 @@ Waiting for completion...
|
|
|
179
200
|
`);
|
|
180
201
|
const result = await job.waitForCompletion(2000, (status) => {
|
|
181
202
|
const progress = status.progress || { percent: '0' };
|
|
182
|
-
const aiStatusDetail = status.summaryStatus
|
|
203
|
+
const aiStatusDetail = status.summaryStatus ? ` [RegressionBot Summary: ${status.summaryStatus}]` : '';
|
|
183
204
|
process.stdout.write(`\r Status: ${status.status} (${progress.percent}%)${aiStatusDetail}`);
|
|
184
205
|
}, { waitForSummaries: !options['skip-summaries'] });
|
|
185
206
|
console.log('\n\n✅ Job Completed.');
|
|
@@ -197,12 +218,7 @@ Waiting for completion...
|
|
|
197
218
|
}
|
|
198
219
|
if (summary.regressionCount > 0) {
|
|
199
220
|
console.log('\n❌ Regressions found:');
|
|
200
|
-
summary.regressions.forEach(
|
|
201
|
-
console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
|
|
202
|
-
console.log(` Diff: ${r.diffUrl}`);
|
|
203
|
-
if (r.regressionbotSummary)
|
|
204
|
-
console.log(` Summary: ${r.regressionbotSummary}`);
|
|
205
|
-
});
|
|
221
|
+
summary.regressions.forEach(printRegression);
|
|
206
222
|
console.log(`\nTo approve these changes, run:\n npx regressionbot approve ${job.jobId}`);
|
|
207
223
|
process.exit(1);
|
|
208
224
|
}
|
|
@@ -237,12 +253,12 @@ Errors: ${summary.errorCount}
|
|
|
237
253
|
`);
|
|
238
254
|
if (summary.regressionCount > 0) {
|
|
239
255
|
console.log('❌ Regressions found:');
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
}
|
|
256
|
+
summary.regressions.forEach(printRegression);
|
|
257
|
+
}
|
|
258
|
+
if (summary.intentAssessment) {
|
|
259
|
+
const ia = summary.intentAssessment;
|
|
260
|
+
console.log(`\n🧭 Intent: ${ia.summary}`);
|
|
261
|
+
console.log(` bugs: ${ia.bugCount}, intentional: ${ia.intentionalCount}, noise: ${ia.noiseCount}, needs review: ${ia.needsReviewCount}`);
|
|
246
262
|
}
|
|
247
263
|
const doDownload = options.download || options['download-full'];
|
|
248
264
|
if (doDownload) {
|
package/dist/index.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary } from './types';
|
|
1
|
+
import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult } from './types';
|
|
2
2
|
import { sanitizeFilename, sanitizeUrlToPath } from './security';
|
|
3
3
|
export { sanitizeFilename, sanitizeUrlToPath, Viewports };
|
|
4
|
-
export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary };
|
|
4
|
+
export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult };
|
|
5
5
|
export declare class RegressionBot {
|
|
6
6
|
private apiKey;
|
|
7
7
|
private apiUrl;
|
|
@@ -40,9 +40,20 @@ export declare class RegressionBot {
|
|
|
40
40
|
options?: any;
|
|
41
41
|
}>;
|
|
42
42
|
masks?: string[];
|
|
43
|
+
/** CSS injected before screenshotting. Max 4096 characters. */
|
|
44
|
+
customCss?: string;
|
|
43
45
|
concurrency?: number;
|
|
44
46
|
autoApprove?: boolean;
|
|
47
|
+
/** What this run is testing, used to judge whether changes were intended. */
|
|
48
|
+
runContext?: RunContext;
|
|
45
49
|
}): Promise<JobHandle>;
|
|
50
|
+
/**
|
|
51
|
+
* Update the configuration settings for a named project.
|
|
52
|
+
*
|
|
53
|
+
* Changing anything that decides what a capture looks like invalidates the
|
|
54
|
+
* stored baselines — see {@link ProjectConfigUpdate}.
|
|
55
|
+
*/
|
|
56
|
+
updateProject(projectName: string, config: ProjectConfigUpdate): Promise<ProjectConfig>;
|
|
46
57
|
_request<T>(path: string, method?: string, body?: any): Promise<T>;
|
|
47
58
|
}
|
|
48
59
|
export declare class JobBuilder {
|
|
@@ -70,6 +81,17 @@ export declare class JobBuilder {
|
|
|
70
81
|
concurrency(n: number): this;
|
|
71
82
|
autoApprove(val?: boolean): this;
|
|
72
83
|
mask(selectors: string[]): this;
|
|
84
|
+
/**
|
|
85
|
+
* Inject custom CSS before each screenshot, e.g. to hide a dynamic widget:
|
|
86
|
+
* `'#chat-widget { display: none !important; }'`. Max 4096 characters.
|
|
87
|
+
*/
|
|
88
|
+
customCss(css: string): this;
|
|
89
|
+
/**
|
|
90
|
+
* Describe what this run is testing — commit, PR, expected changes — so
|
|
91
|
+
* RegressionBot can judge whether each change was intentional.
|
|
92
|
+
* Merges with anything set by an earlier call.
|
|
93
|
+
*/
|
|
94
|
+
withContext(context: RunContext): this;
|
|
73
95
|
/**
|
|
74
96
|
* Compiles the manifest and triggers the API.
|
|
75
97
|
*/
|
|
@@ -81,12 +103,7 @@ export declare class JobHandle {
|
|
|
81
103
|
constructor(sdk: RegressionBot, jobId: string);
|
|
82
104
|
getStatus(): Promise<JobStatus>;
|
|
83
105
|
getSummary(): Promise<JobSummary>;
|
|
84
|
-
approve(): Promise<
|
|
85
|
-
message: string;
|
|
86
|
-
jobId: string;
|
|
87
|
-
approvedUrlsCount: number;
|
|
88
|
-
failedCount?: number;
|
|
89
|
-
}>;
|
|
106
|
+
approve(): Promise<ApproveResult>;
|
|
90
107
|
/**
|
|
91
108
|
* Generate AI summaries on-demand for all regressions in a completed job.
|
|
92
109
|
*/
|
package/dist/index.js
CHANGED
|
@@ -53,6 +53,17 @@ class RegressionBot {
|
|
|
53
53
|
const res = await this._request(`/project/${encodeURIComponent(projectName)}/run`, 'POST', options);
|
|
54
54
|
return new JobHandle(this, res.jobId);
|
|
55
55
|
}
|
|
56
|
+
/**
|
|
57
|
+
* Update the configuration settings for a named project.
|
|
58
|
+
*
|
|
59
|
+
* Changing anything that decides what a capture looks like invalidates the
|
|
60
|
+
* stored baselines — see {@link ProjectConfigUpdate}.
|
|
61
|
+
*/
|
|
62
|
+
async updateProject(projectName, config) {
|
|
63
|
+
// The API wraps the updated project in { message, project }.
|
|
64
|
+
const res = await this._request(`/project/${encodeURIComponent(projectName)}`, 'PUT', config);
|
|
65
|
+
return res.project;
|
|
66
|
+
}
|
|
56
67
|
// Internal fetch wrapper
|
|
57
68
|
async _request(path, method = 'GET', body) {
|
|
58
69
|
const headers = {
|
|
@@ -134,6 +145,23 @@ class JobBuilder {
|
|
|
134
145
|
this.manifest.masks = selectors;
|
|
135
146
|
return this;
|
|
136
147
|
}
|
|
148
|
+
/**
|
|
149
|
+
* Inject custom CSS before each screenshot, e.g. to hide a dynamic widget:
|
|
150
|
+
* `'#chat-widget { display: none !important; }'`. Max 4096 characters.
|
|
151
|
+
*/
|
|
152
|
+
customCss(css) {
|
|
153
|
+
this.manifest.customCss = css;
|
|
154
|
+
return this;
|
|
155
|
+
}
|
|
156
|
+
/**
|
|
157
|
+
* Describe what this run is testing — commit, PR, expected changes — so
|
|
158
|
+
* RegressionBot can judge whether each change was intentional.
|
|
159
|
+
* Merges with anything set by an earlier call.
|
|
160
|
+
*/
|
|
161
|
+
withContext(context) {
|
|
162
|
+
this.manifest.runContext = { ...this.manifest.runContext, ...context };
|
|
163
|
+
return this;
|
|
164
|
+
}
|
|
137
165
|
/**
|
|
138
166
|
* Compiles the manifest and triggers the API.
|
|
139
167
|
*/
|
|
@@ -155,7 +183,9 @@ class JobBuilder {
|
|
|
155
183
|
scans: this.manifest.scans,
|
|
156
184
|
concurrency: this.manifest.concurrency,
|
|
157
185
|
autoApprove: this.manifest.autoApprove,
|
|
158
|
-
masks: this.manifest.masks
|
|
186
|
+
masks: this.manifest.masks,
|
|
187
|
+
customCss: this.manifest.customCss,
|
|
188
|
+
runContext: this.manifest.runContext
|
|
159
189
|
};
|
|
160
190
|
const res = await this.sdk._request('/crawl', 'POST', payload);
|
|
161
191
|
return new JobHandle(this.sdk, res.jobId);
|
|
@@ -197,22 +227,29 @@ class JobHandle {
|
|
|
197
227
|
const jobDir = path.join(baseDir, safeJobId);
|
|
198
228
|
if (!fs.existsSync(jobDir))
|
|
199
229
|
fs.mkdirSync(jobDir, { recursive: true });
|
|
200
|
-
const download = async (url,
|
|
230
|
+
const download = async (url, baseName) => {
|
|
201
231
|
try {
|
|
202
232
|
// 🛡️ SECURITY: Prevent SSRF and local file reads by enforcing HTTP(S) protocol
|
|
203
233
|
(0, security_1.validateProtocol)(url, 'download URL');
|
|
204
234
|
// 🛡️ SECURITY: Fetch with timeout
|
|
205
235
|
const res = await (0, security_1.fetchWithTimeout)(url);
|
|
206
236
|
if (!res.ok) {
|
|
207
|
-
console.warn(`Warning: Failed to download ${
|
|
237
|
+
console.warn(`Warning: Failed to download ${baseName} from ${url} (Status: ${res.status})`);
|
|
208
238
|
return;
|
|
209
239
|
}
|
|
240
|
+
// Determine file extension from Content-Type header
|
|
241
|
+
const contentType = res.headers.get('content-type') || '';
|
|
242
|
+
let ext = '.png'; // default fallback
|
|
243
|
+
if (contentType.includes('image/jpeg') || contentType.includes('image/jpg')) {
|
|
244
|
+
ext = '.jpg';
|
|
245
|
+
}
|
|
246
|
+
const finalName = `${baseName}${ext}`;
|
|
210
247
|
const buffer = Buffer.from(await res.arrayBuffer());
|
|
211
|
-
const filePath = path.join(jobDir,
|
|
248
|
+
const filePath = path.join(jobDir, finalName);
|
|
212
249
|
fs.writeFileSync(filePath, buffer);
|
|
213
250
|
}
|
|
214
251
|
catch (err) {
|
|
215
|
-
console.warn(`Warning: Failed to download ${
|
|
252
|
+
console.warn(`Warning: Failed to download ${baseName}: ${err.message}`);
|
|
216
253
|
}
|
|
217
254
|
};
|
|
218
255
|
// Regressions
|
|
@@ -220,12 +257,12 @@ class JobHandle {
|
|
|
220
257
|
const nameBase = (0, security_1.sanitizeUrlToPath)(r.url);
|
|
221
258
|
const safeVariant = (0, security_1.sanitizeFilename)(r.variantName);
|
|
222
259
|
if (r.diffUrl)
|
|
223
|
-
await download(r.diffUrl, `${nameBase}_diff_${safeVariant}
|
|
260
|
+
await download(r.diffUrl, `${nameBase}_diff_${safeVariant}`);
|
|
224
261
|
if (options.full) {
|
|
225
262
|
if (r.baselineUrl)
|
|
226
|
-
await download(r.baselineUrl, `${nameBase}_baseline_${safeVariant}
|
|
263
|
+
await download(r.baselineUrl, `${nameBase}_baseline_${safeVariant}`);
|
|
227
264
|
if (r.currentUrl)
|
|
228
|
-
await download(r.currentUrl, `${nameBase}_current_${safeVariant}
|
|
265
|
+
await download(r.currentUrl, `${nameBase}_current_${safeVariant}`);
|
|
229
266
|
}
|
|
230
267
|
}
|
|
231
268
|
// Matches (Full only)
|
|
@@ -234,9 +271,9 @@ class JobHandle {
|
|
|
234
271
|
const nameBase = (0, security_1.sanitizeUrlToPath)(m.url);
|
|
235
272
|
const safeVariant = (0, security_1.sanitizeFilename)(m.variantName);
|
|
236
273
|
if (m.baselineUrl)
|
|
237
|
-
await download(m.baselineUrl, `${nameBase}_baseline_${safeVariant}
|
|
274
|
+
await download(m.baselineUrl, `${nameBase}_baseline_${safeVariant}`);
|
|
238
275
|
if (m.currentUrl)
|
|
239
|
-
await download(m.currentUrl, `${nameBase}_current_${safeVariant}
|
|
276
|
+
await download(m.currentUrl, `${nameBase}_current_${safeVariant}`);
|
|
240
277
|
}
|
|
241
278
|
}
|
|
242
279
|
}
|
package/dist/types.d.ts
CHANGED
|
@@ -9,23 +9,137 @@ export interface ProjectScan {
|
|
|
9
9
|
exclude?: string[];
|
|
10
10
|
};
|
|
11
11
|
}
|
|
12
|
+
/**
|
|
13
|
+
* Credentials for reaching an origin behind an environment gate — basic auth,
|
|
14
|
+
* a bypass header, or a session cookie. Write-only: the API never returns these,
|
|
15
|
+
* only a `{ configured: true }` presence flag.
|
|
16
|
+
*/
|
|
17
|
+
export interface EnvGate {
|
|
18
|
+
basic?: {
|
|
19
|
+
username: string;
|
|
20
|
+
password: string;
|
|
21
|
+
};
|
|
22
|
+
headers?: Record<string, string>;
|
|
23
|
+
cookies?: Array<{
|
|
24
|
+
name: string;
|
|
25
|
+
value: string;
|
|
26
|
+
}>;
|
|
27
|
+
}
|
|
28
|
+
/** How a project's baseline advances between runs. Absent means `approved`. */
|
|
29
|
+
export type BaselinePolicy = 'approved' | 'rolling';
|
|
30
|
+
/** Frequency at which a project runs unattended. */
|
|
31
|
+
export type ProjectSchedule = 'hourly' | 'daily' | 'weekly';
|
|
32
|
+
/**
|
|
33
|
+
* A saved project as the API returns it.
|
|
34
|
+
*
|
|
35
|
+
* This is a read shape. To change a project, pass {@link ProjectConfigUpdate}
|
|
36
|
+
* to updateProject() — the two differ, because gate credentials are written as
|
|
37
|
+
* an {@link EnvGate} but read back only as a presence flag.
|
|
38
|
+
*/
|
|
12
39
|
export interface ProjectConfig {
|
|
13
|
-
orgId?: string;
|
|
14
40
|
name: string;
|
|
15
|
-
testOrigin
|
|
41
|
+
testOrigin: string;
|
|
16
42
|
baseOrigin?: string;
|
|
17
43
|
sitemapUrl?: string;
|
|
18
44
|
paths?: ProjectPath[];
|
|
19
45
|
scans?: ProjectScan[];
|
|
20
|
-
devices
|
|
46
|
+
devices: string[];
|
|
21
47
|
masks?: string[];
|
|
48
|
+
/** CSS injected before screenshotting, to hide dynamic widgets. Max 4096 characters. */
|
|
49
|
+
customCss?: string;
|
|
50
|
+
/** Presence flag only — the stored credential is never returned. */
|
|
51
|
+
testAuth?: {
|
|
52
|
+
configured: true;
|
|
53
|
+
};
|
|
54
|
+
/** Presence flag only — the stored credential is never returned. */
|
|
55
|
+
baseAuth?: {
|
|
56
|
+
configured: true;
|
|
57
|
+
};
|
|
22
58
|
concurrency?: number;
|
|
23
|
-
|
|
24
|
-
|
|
59
|
+
/** Extra instructions handed to the model that writes change summaries. */
|
|
60
|
+
aiPromptInstructions?: string;
|
|
61
|
+
/** Diff percentage below which a regression is skipped by the AI pass. Defaults to 0.01. */
|
|
62
|
+
aiSummaryThreshold?: number;
|
|
63
|
+
/**
|
|
64
|
+
* `approved` (default): a person accepts a run and its captures become the baseline.
|
|
65
|
+
* `rolling`: every completed managed run advances it, and those runs auto-approve.
|
|
66
|
+
*/
|
|
67
|
+
baselinePolicy?: BaselinePolicy;
|
|
68
|
+
/** Set when the project runs unattended. Absent means it only runs when triggered. */
|
|
69
|
+
schedule?: ProjectSchedule;
|
|
70
|
+
/**
|
|
71
|
+
* The UTC hour, 0–23, at which a `daily` or `weekly` project runs — `3` is 03:00 UTC.
|
|
72
|
+
* Absent means the slot anchors to whenever the first sweep picked the project up.
|
|
73
|
+
*/
|
|
74
|
+
scheduleHourUtc?: number;
|
|
75
|
+
/**
|
|
76
|
+
* The API key recorded when the schedule was set; every scheduled run is attributed
|
|
77
|
+
* to it. Absent if the schedule was set without an API key, in which case AI
|
|
78
|
+
* summaries are skipped for those runs.
|
|
79
|
+
*/
|
|
80
|
+
scheduleKeyId?: string;
|
|
81
|
+
/** When the scheduler last started a run. The next run is due one interval after this. */
|
|
82
|
+
lastScheduledRunAt?: string;
|
|
83
|
+
createdAt: string;
|
|
84
|
+
updatedAt: string;
|
|
25
85
|
lastRunAt?: string;
|
|
26
86
|
lastJobId?: string;
|
|
87
|
+
baselineVersion?: string;
|
|
27
88
|
baselineInvalidatedAt?: string;
|
|
28
89
|
}
|
|
90
|
+
/**
|
|
91
|
+
* The fields updateProject() accepts.
|
|
92
|
+
*
|
|
93
|
+
* Changing anything that decides what a capture looks like — testOrigin,
|
|
94
|
+
* baseOrigin, sitemapUrl, paths, scans, devices, masks, customCss, or a gate
|
|
95
|
+
* credential — invalidates the stored baselines, and the next run re-captures
|
|
96
|
+
* them instead of comparing. Everything else leaves baselines alone.
|
|
97
|
+
* Re-sending an identical value is not a change.
|
|
98
|
+
*/
|
|
99
|
+
export interface ProjectConfigUpdate {
|
|
100
|
+
testOrigin?: string;
|
|
101
|
+
baseOrigin?: string;
|
|
102
|
+
sitemapUrl?: string;
|
|
103
|
+
paths?: ProjectPath[];
|
|
104
|
+
scans?: ProjectScan[];
|
|
105
|
+
devices?: string[];
|
|
106
|
+
masks?: string[];
|
|
107
|
+
/** Max 4096 characters. */
|
|
108
|
+
customCss?: string;
|
|
109
|
+
/** Credentials for the test origin's environment gate. Pass null to clear. */
|
|
110
|
+
testAuth?: EnvGate | null;
|
|
111
|
+
/** Credentials for the base origin's environment gate. Pass null to clear. */
|
|
112
|
+
baseAuth?: EnvGate | null;
|
|
113
|
+
concurrency?: number;
|
|
114
|
+
aiPromptInstructions?: string;
|
|
115
|
+
aiSummaryThreshold?: number;
|
|
116
|
+
baselinePolicy?: BaselinePolicy;
|
|
117
|
+
/**
|
|
118
|
+
* Run this project unattended at this frequency. Requires `baselinePolicy: 'rolling'`
|
|
119
|
+
* on a managed project — the API rejects the combination without it, because on the
|
|
120
|
+
* approved policy an unapproved change is re-reported on every subsequent run.
|
|
121
|
+
* Live-vs-live projects store no baseline and are exempt. Pass null to remove.
|
|
122
|
+
*
|
|
123
|
+
* Without `scheduleHourUtc` the first run starts at the next hourly sweep and the
|
|
124
|
+
* cadence anchors to it.
|
|
125
|
+
*/
|
|
126
|
+
schedule?: ProjectSchedule | null;
|
|
127
|
+
/**
|
|
128
|
+
* Pin a `daily` or `weekly` schedule to this UTC hour, 0–23 — `3` means 03:00 UTC.
|
|
129
|
+
* The first run waits for the hour too, rather than firing immediately and settling
|
|
130
|
+
* into the hour from the following day. Hours only, because the scheduler sweeps once
|
|
131
|
+
* an hour; UTC only, because RegressionBot has no timezone to convert from.
|
|
132
|
+
*
|
|
133
|
+
* Rejected on an `hourly` schedule, which already runs every hour, and rejected if the
|
|
134
|
+
* project has no schedule to apply it to — either one already set, or one sent in the
|
|
135
|
+
* same update. Clearing the schedule with
|
|
136
|
+
* `schedule: null` clears this too, so re-enabling later cannot resurrect an hour
|
|
137
|
+
* nobody asked for on that run. Pass null to clear it on its own.
|
|
138
|
+
*
|
|
139
|
+
* A missed slot waits for the next one rather than catching up.
|
|
140
|
+
*/
|
|
141
|
+
scheduleHourUtc?: number | null;
|
|
142
|
+
}
|
|
29
143
|
export interface VRConfig {
|
|
30
144
|
apiKey?: string;
|
|
31
145
|
apiUrl?: string;
|
|
@@ -52,53 +166,105 @@ export declare const Viewports: {
|
|
|
52
166
|
readonly height: 667;
|
|
53
167
|
};
|
|
54
168
|
};
|
|
169
|
+
export type VerdictDecision = 'intentional' | 'bug' | 'noise' | 'needs_review';
|
|
170
|
+
/** How one labelled region of a diff was judged against the run's stated intent. */
|
|
171
|
+
export interface RegionVerdict {
|
|
172
|
+
decision: VerdictDecision;
|
|
173
|
+
/** 0–1. */
|
|
174
|
+
confidence: number;
|
|
175
|
+
reasoning: string;
|
|
176
|
+
}
|
|
177
|
+
/** The rolled-up verdict for one result, plus the per-region verdicts behind it. */
|
|
178
|
+
export interface ResultVerdict {
|
|
179
|
+
decision: VerdictDecision;
|
|
180
|
+
minConfidence: number;
|
|
181
|
+
avgConfidence: number;
|
|
182
|
+
/** Keyed by region label (A, B, C…). Empty when the verdict came from the text-only pass. */
|
|
183
|
+
regions: Record<string, RegionVerdict>;
|
|
184
|
+
}
|
|
185
|
+
export interface RegressionbotSummaryItem {
|
|
186
|
+
/** Region letter (A, B, C…), or an empty string for a whole-page change. */
|
|
187
|
+
label: string;
|
|
188
|
+
/** Single-sentence description of what changed in that region. */
|
|
189
|
+
text: string;
|
|
190
|
+
verdict?: RegionVerdict;
|
|
191
|
+
}
|
|
55
192
|
/**
|
|
56
193
|
* Result for a single URL + device comparison.
|
|
57
194
|
*
|
|
58
|
-
* Returned by
|
|
59
|
-
* (
|
|
60
|
-
*
|
|
195
|
+
* Returned by getStatus() and, split across `regressions`/`matches`, by
|
|
196
|
+
* getSummary(). Both endpoints pre-sign the image URLs; a URL is null when
|
|
197
|
+
* that image does not exist for this result (no baseline yet, no diff, …).
|
|
61
198
|
*/
|
|
62
199
|
export interface PageResult {
|
|
63
200
|
url: string;
|
|
64
|
-
/**
|
|
201
|
+
/** Whether the capture and comparison succeeded. */
|
|
202
|
+
status: 'SUCCESS' | 'ERROR';
|
|
203
|
+
/** Device variant tested (e.g. "Desktop Chrome", "iPhone 12"). */
|
|
65
204
|
variantName: string;
|
|
66
205
|
/** Percentage of pixels that differed from the baseline (0 = identical). */
|
|
67
206
|
diffPercentage: number;
|
|
68
|
-
/** Raw count of differing pixels. */
|
|
69
|
-
diffCount: number;
|
|
70
207
|
/** Perceptual similarity 0-100 (SSIM). */
|
|
71
208
|
visualMatchScore: number;
|
|
209
|
+
/** True when this capture became a baseline because none existed to compare against. */
|
|
210
|
+
isNewBaseline: boolean;
|
|
211
|
+
/** Why the capture or comparison failed. Present when status is ERROR. */
|
|
212
|
+
errorMessage?: string;
|
|
72
213
|
/** Plain-English description of what visually changed, generated by RegressionBot. */
|
|
73
|
-
regressionbotSummary?:
|
|
214
|
+
regressionbotSummary?: RegressionbotSummaryItem[];
|
|
215
|
+
/**
|
|
216
|
+
* Whether the change was judged intentional, a bug, or noise. Only produced when
|
|
217
|
+
* the run carried a {@link RunContext}, and only once summaryStatus is COMPLETE.
|
|
218
|
+
*/
|
|
219
|
+
verdict?: ResultVerdict;
|
|
220
|
+
/** Selectors of the elements that changed, when the DOM comparison could identify them. */
|
|
221
|
+
elementsChanged?: string[];
|
|
222
|
+
/**
|
|
223
|
+
* Set when the DOM comparison could not run. The result is a plain pixel diff with
|
|
224
|
+
* no semantic change types, so treat its classifications as absent rather than as
|
|
225
|
+
* "nothing structural changed".
|
|
226
|
+
*/
|
|
227
|
+
domAssistSkipReason?: string;
|
|
74
228
|
/** Pre-signed URL for the stored baseline screenshot. */
|
|
75
|
-
baselineUrl
|
|
229
|
+
baselineUrl: string | null;
|
|
76
230
|
/** Pre-signed URL for the newly captured screenshot. */
|
|
77
|
-
currentUrl
|
|
231
|
+
currentUrl: string | null;
|
|
78
232
|
/** Pre-signed URL for the side-by-side annotated diff image. */
|
|
79
|
-
diffUrl
|
|
233
|
+
diffUrl: string | null;
|
|
234
|
+
/**
|
|
235
|
+
* Pre-signed URL for the diff mask. getStatus() always sends the key, null when
|
|
236
|
+
* there is no mask; getSummary() omits it entirely in that case.
|
|
237
|
+
*/
|
|
238
|
+
maskUrl?: string | null;
|
|
80
239
|
}
|
|
81
240
|
export interface JobProgress {
|
|
82
241
|
total: number;
|
|
83
242
|
completed: number;
|
|
84
243
|
percent: string;
|
|
85
244
|
}
|
|
245
|
+
/** Progress of the asynchronous AI summary pass. */
|
|
246
|
+
export type SummaryStatus = 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'FAILED';
|
|
86
247
|
export interface JobStatus {
|
|
87
248
|
jobId: string;
|
|
88
249
|
status: 'INITIALIZING' | 'PROCESSING' | 'FINISHING' | 'SUMMARIZING' | 'COMPLETED' | 'APPROVED' | 'FAILED';
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
250
|
+
/**
|
|
251
|
+
* PENDING means not yet started, COMPLETE that regressionbotSummary is populated
|
|
252
|
+
* (or that nothing needed one). Poll until COMPLETE before reading summaries.
|
|
253
|
+
*/
|
|
254
|
+
summaryStatus: SummaryStatus;
|
|
255
|
+
error: string | null;
|
|
256
|
+
progress: JobProgress;
|
|
257
|
+
executionTime: number;
|
|
258
|
+
createdAt: string;
|
|
259
|
+
/** Partial or complete results. Fills in as workers finish. */
|
|
260
|
+
results: PageResult[];
|
|
96
261
|
}
|
|
97
262
|
export interface JobSummary {
|
|
98
263
|
jobId: string;
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
264
|
+
/** getSummary() throws until the job reaches one of these. */
|
|
265
|
+
status: 'COMPLETED' | 'APPROVED' | 'FAILED';
|
|
266
|
+
summaryStatus: SummaryStatus;
|
|
267
|
+
error: string | null;
|
|
102
268
|
totalUrls: number;
|
|
103
269
|
completedCount: number;
|
|
104
270
|
/** Overall quality score 0–100 across all tested pages. */
|
|
@@ -108,9 +274,9 @@ export interface JobSummary {
|
|
|
108
274
|
matchCount: number;
|
|
109
275
|
newBaselineCount: number;
|
|
110
276
|
errorCount: number;
|
|
111
|
-
/** Pages where a visual regression was detected.
|
|
277
|
+
/** Pages where a visual regression was detected. */
|
|
112
278
|
regressions: PageResult[];
|
|
113
|
-
/** Pages that matched the baseline.
|
|
279
|
+
/** Pages that matched the baseline. */
|
|
114
280
|
matches: PageResult[];
|
|
115
281
|
/** Pages captured for the first time — no baseline existed to compare against. */
|
|
116
282
|
newBaselines: Array<{
|
|
@@ -123,6 +289,49 @@ export interface JobSummary {
|
|
|
123
289
|
variantName: string;
|
|
124
290
|
errorMessage: string;
|
|
125
291
|
}>;
|
|
292
|
+
/** Whole-job roll-up of how the changes line up with the run's stated intent. */
|
|
293
|
+
intentAssessment: IntentAssessment;
|
|
294
|
+
/** The intent this run was given, echoed back. */
|
|
295
|
+
runContext?: RunContext;
|
|
296
|
+
}
|
|
297
|
+
/**
|
|
298
|
+
* What a run is testing. Supplying it lets RegressionBot judge each change as
|
|
299
|
+
* intentional, a bug, or noise, rather than just reporting that pixels moved.
|
|
300
|
+
*/
|
|
301
|
+
export interface RunContext {
|
|
302
|
+
/** Short description of what this run is testing. */
|
|
303
|
+
changeDescription?: string;
|
|
304
|
+
gitCommitSha?: string;
|
|
305
|
+
/** Git commit subject line. */
|
|
306
|
+
gitCommitMessage?: string;
|
|
307
|
+
prTitle?: string;
|
|
308
|
+
/** Markdown. Truncated to 2000 characters by the API. */
|
|
309
|
+
prDescription?: string;
|
|
310
|
+
/** Explicit list of expected visual changes. */
|
|
311
|
+
expectedChanges?: string[];
|
|
312
|
+
/** Files or component names changed. */
|
|
313
|
+
scope?: string[];
|
|
314
|
+
}
|
|
315
|
+
export interface IntentAssessment {
|
|
316
|
+
/** False when the run carried no RunContext, in which case the counts are all zero. */
|
|
317
|
+
intentProvided: boolean;
|
|
318
|
+
bugCount: number;
|
|
319
|
+
intentionalCount: number;
|
|
320
|
+
noiseCount: number;
|
|
321
|
+
needsReviewCount: number;
|
|
322
|
+
/** True when every regression is accounted for by the stated intent. */
|
|
323
|
+
allAccountedFor: boolean;
|
|
324
|
+
/** One line summarising the job, safe to print straight into CI output. */
|
|
325
|
+
summary: string;
|
|
326
|
+
}
|
|
327
|
+
export interface ApproveResult {
|
|
328
|
+
message: string;
|
|
329
|
+
jobId: string;
|
|
330
|
+
approvedUrlsCount: number;
|
|
331
|
+
/** Present when some captures could not be promoted. */
|
|
332
|
+
failedCount?: number;
|
|
333
|
+
/** Pages skipped because another job updated their baseline first. */
|
|
334
|
+
conflictedUrls?: string[];
|
|
126
335
|
}
|
|
127
336
|
export interface JobAiSummary {
|
|
128
337
|
message: string;
|
|
@@ -130,6 +339,6 @@ export interface JobAiSummary {
|
|
|
130
339
|
summaries: Array<{
|
|
131
340
|
url: string;
|
|
132
341
|
variantName: string;
|
|
133
|
-
regressionbotSummary:
|
|
342
|
+
regressionbotSummary: RegressionbotSummaryItem[];
|
|
134
343
|
}>;
|
|
135
344
|
}
|
package/package.json
CHANGED