@regressionbot/sdk 1.2.3 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +181 -4
- package/dist/cli.js +29 -13
- package/dist/index.d.ts +22 -9
- package/dist/index.js +42 -11
- package/dist/types.d.ts +231 -26
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -13,7 +13,7 @@ RegressionBot is a declarative visual regression testing platform that helps you
|
|
|
13
13
|
|
|
14
14
|
Unlike traditional visual diffing libraries, RegressionBot is designed for modern, automated development loops and agentic pipelines:
|
|
15
15
|
|
|
16
|
-
- **Highly Accurate Regressions (
|
|
16
|
+
- **Highly Accurate Regressions (Less Noise)**: Leveraging advanced pixel-matching algorithms and element masking (using CSS selectors or the automatic `data-vr-mask` attribute), RegressionBot minimizes false positives caused by dynamic data, layout shifting, or third-party widgets.
|
|
17
17
|
- **Plain-English Summaries**: No more manual screenshot comparisons. RegressionBot translates visual diffs into concise, plain-English descriptions of what changed, so you know exactly what was modified at a glance.
|
|
18
18
|
- **Agentic Workflow Ready**: Built from the ground up to support autonomous coding agents (like Gemini or Claude) and automated developer loops. Through standard API endpoints, CLI commands, and Model Context Protocol (MCP) integrations, agents can trigger tests, read plain-English results, and approve baseline changes programmatically without human intervention.
|
|
19
19
|
|
|
@@ -24,6 +24,8 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
|
|
|
24
24
|
- **Matrix Testing**: Test multiple devices and viewports in a single job.
|
|
25
25
|
- **Auto-Discovery**: Scan sitemaps with glob patterns and limits.
|
|
26
26
|
- **RegressionBot Summaries**: Plain-English change descriptions for every regression, generated on-demand via the API.
|
|
27
|
+
- **Intent-Aware Verdicts**: Describe what a run is meant to change, and every regression comes back judged as intentional, a bug, or noise.
|
|
28
|
+
- **Scheduled Checks**: Run a saved project unattended, hourly, daily, or weekly — and at a set UTC hour.
|
|
27
29
|
- **Project-Based Baselines**: Save and reuse test configurations; share visual history across environments.
|
|
28
30
|
- **Auto-Approval**: Automatically promote screenshots to baselines on jobs that pass your criteria.
|
|
29
31
|
- **Zero Infrastructure**: No browser maintenance or server provisioning — RegressionBot handles it all.
|
|
@@ -34,6 +36,44 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
|
|
|
34
36
|
npm install @regressionbot/sdk
|
|
35
37
|
```
|
|
36
38
|
|
|
39
|
+
## Migration from 1.x
|
|
40
|
+
|
|
41
|
+
2.0.0 re-types the SDK against what the API actually returns, taken from the
|
|
42
|
+
orchestrator's implementation rather than from a spec that had drifted in both
|
|
43
|
+
directions. Most of the breakage is caught by the compiler. One is not:
|
|
44
|
+
|
|
45
|
+
```typescript
|
|
46
|
+
// 1.x — updateProject() returned the raw { message, project } envelope,
|
|
47
|
+
// so every field read off it was undefined at runtime
|
|
48
|
+
const project = await rb.updateProject('site', { schedule: 'daily' });
|
|
49
|
+
project.project.schedule; // this is where the value actually was
|
|
50
|
+
|
|
51
|
+
// 2.0 — the envelope is unwrapped for you
|
|
52
|
+
const project = await rb.updateProject('site', { schedule: 'daily' });
|
|
53
|
+
project.schedule; // 'daily'
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
The rest surface as type errors:
|
|
57
|
+
|
|
58
|
+
- **`PageResult.diffCount` is gone.** No endpoint ever returned it — both
|
|
59
|
+
`getJobStatus` and `getJobSummary` strip it. `PageResult` gains the fields they
|
|
60
|
+
do return: `status`, `isNewBaseline`, `errorMessage`, `elementsChanged`,
|
|
61
|
+
`domAssistSkipReason`, `maskUrl` and `verdict`. Image URLs are typed nullable,
|
|
62
|
+
which they always were.
|
|
63
|
+
- **`updateProject()` takes `ProjectConfigUpdate`, not `Partial<ProjectConfig>`.**
|
|
64
|
+
The read and write shapes genuinely differ: a gate credential is written as an
|
|
65
|
+
`EnvGate` and read back only as `{ configured: true }`.
|
|
66
|
+
- **`ProjectConfig.orgId` is gone** — the API stopped returning it.
|
|
67
|
+
- **`summaryStatus` can be `FAILED`**, which the old union omitted. `error`,
|
|
68
|
+
`progress`, `results`, `createdAt` and `intentAssessment` are always returned.
|
|
69
|
+
- **`approve()` gains `conflictedUrls`**, returned when another job updated a
|
|
70
|
+
baseline first.
|
|
71
|
+
|
|
72
|
+
New in 2.0 and requiring a current API: `.customCss()` and `.withContext()` on
|
|
73
|
+
the builder, intent-aware verdicts, environment gates, baseline policies, and
|
|
74
|
+
scheduling. `scheduleHourUtc` needs API 2.7.0 or later — on an older API the
|
|
75
|
+
field is accepted and ignored, and the schedule stays anchored to its first run.
|
|
76
|
+
|
|
37
77
|
## Usage
|
|
38
78
|
|
|
39
79
|
### Basic Example
|
|
@@ -91,6 +131,12 @@ const job = await rb
|
|
|
91
131
|
.mask(['.ads', '#modal']) // Manual selectors
|
|
92
132
|
// Tip: Adding 'data-vr-mask' to your HTML elements masks them automatically!
|
|
93
133
|
|
|
134
|
+
// Custom CSS: Injected before every screenshot (max 4096 chars)
|
|
135
|
+
.customCss('#chat-widget { display: none !important; }')
|
|
136
|
+
|
|
137
|
+
// Intent: What this run is testing, so changes can be judged against it
|
|
138
|
+
.withContext({ changeDescription: 'Restyle the pricing table' })
|
|
139
|
+
|
|
94
140
|
// Execute: Compiles manifest and triggers the API
|
|
95
141
|
.run();
|
|
96
142
|
|
|
@@ -112,9 +158,9 @@ const summary = await job.getSummary();
|
|
|
112
158
|
if (summary.regressions.length > 0) {
|
|
113
159
|
console.log(`\n${summary.regressions.length} regressions found:`);
|
|
114
160
|
for (const regression of summary.regressions) {
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
console.log(
|
|
161
|
+
for (const item of regression.regressionbotSummary ?? []) {
|
|
162
|
+
// `label` is the region letter (A, B, C…), or '' for a whole-page change
|
|
163
|
+
console.log(`[${item.label}] ${item.text}`);
|
|
118
164
|
}
|
|
119
165
|
}
|
|
120
166
|
}
|
|
@@ -124,6 +170,68 @@ const aiResult = await job.generateAiSummary();
|
|
|
124
170
|
console.log(`Generated summaries for ${aiResult.summaries.length} regressions.`);
|
|
125
171
|
```
|
|
126
172
|
|
|
173
|
+
### Intent-Aware Verdicts
|
|
174
|
+
|
|
175
|
+
Tell RegressionBot what a run is meant to change, and every regression comes back judged
|
|
176
|
+
against that intent — `intentional`, `bug`, `noise`, or `needs_review`.
|
|
177
|
+
|
|
178
|
+
```typescript
|
|
179
|
+
const job = await rb
|
|
180
|
+
.test(process.env.VERCEL_PREVIEW_URL)
|
|
181
|
+
.forProject('marketing-site-v2')
|
|
182
|
+
.withContext({
|
|
183
|
+
changeDescription: 'Restyle the pricing table',
|
|
184
|
+
gitCommitSha: process.env.GITHUB_SHA,
|
|
185
|
+
prTitle: process.env.PR_TITLE,
|
|
186
|
+
expectedChanges: ['Pricing table background is now dark'],
|
|
187
|
+
scope: ['components/PricingTable.tsx'],
|
|
188
|
+
})
|
|
189
|
+
.run();
|
|
190
|
+
|
|
191
|
+
await job.waitForCompletion(2000, undefined, { waitForSummaries: true });
|
|
192
|
+
const summary = await job.getSummary();
|
|
193
|
+
|
|
194
|
+
// Whole-job roll-up: does everything that changed line up with the stated intent?
|
|
195
|
+
console.log(summary.intentAssessment?.summary);
|
|
196
|
+
console.log(`Bugs: ${summary.intentAssessment?.bugCount}`);
|
|
197
|
+
|
|
198
|
+
// Per-result verdict, plus the per-region verdicts behind it
|
|
199
|
+
for (const r of summary.regressions) {
|
|
200
|
+
console.log(`${r.url}: ${r.verdict?.decision} (${r.verdict?.avgConfidence})`);
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
// Fail CI only on changes that were not intended
|
|
204
|
+
const bugs = summary.regressions.filter(r => r.verdict?.decision === 'bug');
|
|
205
|
+
if (bugs.length > 0) process.exit(1);
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
`runProject()` takes the same context, so a run from a saved project can be
|
|
209
|
+
intent-aware too:
|
|
210
|
+
|
|
211
|
+
```typescript
|
|
212
|
+
const job = await rb.runProject('marketing-site-v2', {
|
|
213
|
+
testOrigin: process.env.VERCEL_PREVIEW_URL,
|
|
214
|
+
runContext: { changeDescription: 'Restyle the pricing table' },
|
|
215
|
+
});
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
### Hiding Dynamic Content
|
|
219
|
+
|
|
220
|
+
`.mask()` hides elements by selector. `.customCss()` injects arbitrary CSS before each
|
|
221
|
+
screenshot when masking isn't enough — collapsing an animation, pinning a carousel, or
|
|
222
|
+
neutralising a third-party widget. Max 4096 characters.
|
|
223
|
+
|
|
224
|
+
```typescript
|
|
225
|
+
const job = await rb
|
|
226
|
+
.test('https://preview.myapp.com')
|
|
227
|
+
.forProject('my-app-web')
|
|
228
|
+
.customCss(`
|
|
229
|
+
#chat-widget { display: none !important; }
|
|
230
|
+
* { animation: none !important; transition: none !important; }
|
|
231
|
+
`)
|
|
232
|
+
.run();
|
|
233
|
+
```
|
|
234
|
+
|
|
127
235
|
### Progress Tracking
|
|
128
236
|
|
|
129
237
|
Pass a callback to `waitForCompletion` to receive status updates while the job runs:
|
|
@@ -177,6 +285,69 @@ const summary = await job.getSummary();
|
|
|
177
285
|
console.log(`Score: ${summary.overallScore}/100`);
|
|
178
286
|
```
|
|
179
287
|
|
|
288
|
+
### Environment Gates
|
|
289
|
+
|
|
290
|
+
If an origin sits behind basic auth, a bypass header, or a session cookie, store the
|
|
291
|
+
credential on the project. It is encrypted at rest and never returned — reads report
|
|
292
|
+
only `{ configured: true }`.
|
|
293
|
+
|
|
294
|
+
```typescript
|
|
295
|
+
await rb.updateProject('marketing-site-v2', {
|
|
296
|
+
testAuth: { basic: { username: 'preview', password: process.env.PREVIEW_PASSWORD! } },
|
|
297
|
+
// or: { headers: { 'x-vercel-protection-bypass': token } }
|
|
298
|
+
// or: { cookies: [{ name: 'session', value: token }] }
|
|
299
|
+
});
|
|
300
|
+
|
|
301
|
+
const project = await rb.getProject('marketing-site-v2');
|
|
302
|
+
console.log(project.testAuth); // { configured: true }
|
|
303
|
+
|
|
304
|
+
// Clear it
|
|
305
|
+
await rb.updateProject('marketing-site-v2', { testAuth: null });
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
Setting or clearing a gate credential invalidates the stored baselines, since it changes
|
|
309
|
+
what the capture can reach.
|
|
310
|
+
|
|
311
|
+
### Scheduled Checks
|
|
312
|
+
|
|
313
|
+
A project can run unattended on a fixed cadence. Scheduling requires
|
|
314
|
+
`baselinePolicy: 'rolling'` on a managed project — on the default `approved` policy an
|
|
315
|
+
unapproved change would be re-reported on every subsequent run, so the API rejects the
|
|
316
|
+
combination. Live-vs-live projects store no baseline and are exempt.
|
|
317
|
+
|
|
318
|
+
```typescript
|
|
319
|
+
// Turn on a daily unattended check
|
|
320
|
+
await rb.updateProject('marketing-site-v2', {
|
|
321
|
+
baselinePolicy: 'rolling',
|
|
322
|
+
schedule: 'daily', // 'hourly' | 'daily' | 'weekly'
|
|
323
|
+
});
|
|
324
|
+
|
|
325
|
+
// Or pin it to a UTC hour — this one runs at 03:00 UTC every day
|
|
326
|
+
await rb.updateProject('marketing-site-v2', {
|
|
327
|
+
baselinePolicy: 'rolling',
|
|
328
|
+
schedule: 'daily',
|
|
329
|
+
scheduleHourUtc: 3, // 0–23, UTC
|
|
330
|
+
});
|
|
331
|
+
|
|
332
|
+
// Turn it back off (this clears scheduleHourUtc too)
|
|
333
|
+
await rb.updateProject('marketing-site-v2', { schedule: null });
|
|
334
|
+
|
|
335
|
+
// Read back when the scheduler last fired — the next run is due one interval after this
|
|
336
|
+
const project = await rb.getProject('marketing-site-v2');
|
|
337
|
+
console.log(project.schedule, project.scheduleHourUtc, project.lastScheduledRunAt);
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
Without `scheduleHourUtc` the first run starts at the next hourly sweep and the cadence
|
|
341
|
+
anchors to it. With it, the first run waits for that hour as well. Hours only, since the
|
|
342
|
+
scheduler sweeps once an hour, and UTC only. It is rejected on an `hourly` schedule, and
|
|
343
|
+
rejected on a project with no schedule to apply it to. Clearing the schedule clears the
|
|
344
|
+
hour with it, and a missed slot waits for the next one rather than catching up.
|
|
345
|
+
|
|
346
|
+
Setting a schedule, its hour, or a baseline policy does **not** invalidate
|
|
347
|
+
baselines — only changes that affect what a capture looks like (`testOrigin`,
|
|
348
|
+
`baseOrigin`, `sitemapUrl`, `paths`, `scans`, `devices`, `masks`, `customCss`) do that.
|
|
349
|
+
Billing is per comparison, so cost scales with frequency.
|
|
350
|
+
|
|
180
351
|
### Reconnecting to an Existing Job
|
|
181
352
|
|
|
182
353
|
If you have a job ID from a previous run (e.g., stored in CI state), you can attach to it without re-running the test:
|
|
@@ -210,6 +381,12 @@ Test an entire site using glob patterns.
|
|
|
210
381
|
npx @regressionbot/sdk https://example.com --project my-project --scan "/**" --exclude "/admin/**" --concurrency 20
|
|
211
382
|
```
|
|
212
383
|
|
|
384
|
+
Use `--mask` to hide elements by selector, or `--custom-css` to inject CSS before every
|
|
385
|
+
screenshot:
|
|
386
|
+
```bash
|
|
387
|
+
npx @regressionbot/sdk https://example.com --project my-site --custom-css "#chat-widget { display: none !important; }"
|
|
388
|
+
```
|
|
389
|
+
|
|
213
390
|
#### 3. Job Summary
|
|
214
391
|
Get detailed results and diff URLs for a completed job.
|
|
215
392
|
```bash
|
package/dist/cli.js
CHANGED
|
@@ -37,6 +37,23 @@ Object.defineProperty(exports, "__esModule", { value: true });
|
|
|
37
37
|
const index_1 = require("./index");
|
|
38
38
|
const security_1 = require("./security");
|
|
39
39
|
const path = __importStar(require("path"));
|
|
40
|
+
function formatSummary(items) {
|
|
41
|
+
if (!items || items.length === 0)
|
|
42
|
+
return '';
|
|
43
|
+
return items
|
|
44
|
+
.map(item => (item.label ? `[${item.label}] ${item.text}` : item.text))
|
|
45
|
+
.join('\n ');
|
|
46
|
+
}
|
|
47
|
+
function printRegression(r) {
|
|
48
|
+
console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
|
|
49
|
+
console.log(` Diff: ${r.diffUrl}`);
|
|
50
|
+
if (r.verdict) {
|
|
51
|
+
console.log(` Verdict: ${r.verdict.decision} (confidence ${r.verdict.minConfidence.toFixed(2)}-${r.verdict.avgConfidence.toFixed(2)})`);
|
|
52
|
+
}
|
|
53
|
+
const summary = formatSummary(r.regressionbotSummary);
|
|
54
|
+
if (summary)
|
|
55
|
+
console.log(` Summary: ${summary}`);
|
|
56
|
+
}
|
|
40
57
|
function parseArgs(args) {
|
|
41
58
|
const options = Object.create(null);
|
|
42
59
|
options._ = [];
|
|
@@ -129,6 +146,7 @@ Options for <url>:
|
|
|
129
146
|
--concurrency <n> Max concurrent workers (default 10).
|
|
130
147
|
--auto-approve Automatically approve results as new baselines.
|
|
131
148
|
--mask <selectors> Comma-separated CSS selectors to hide (e.g. ".ad,#popup").
|
|
149
|
+
--custom-css <css> CSS injected before each screenshot (max 4096 chars).
|
|
132
150
|
--skip-summaries Skip waiting for parallel AI summaries in the CLI.
|
|
133
151
|
|
|
134
152
|
Environment Variables:
|
|
@@ -167,6 +185,9 @@ async function startJob(url, options) {
|
|
|
167
185
|
const selectors = options.mask.split(',').map((s) => s.trim());
|
|
168
186
|
builder.mask(selectors);
|
|
169
187
|
}
|
|
188
|
+
if (typeof options['custom-css'] === 'string') {
|
|
189
|
+
builder.customCss(options['custom-css']);
|
|
190
|
+
}
|
|
170
191
|
const job = await builder.run();
|
|
171
192
|
console.log(`✅ Job started! ID: ${job.jobId}`);
|
|
172
193
|
console.log(`📊 Project: ${projectId}`);
|
|
@@ -179,7 +200,7 @@ Waiting for completion...
|
|
|
179
200
|
`);
|
|
180
201
|
const result = await job.waitForCompletion(2000, (status) => {
|
|
181
202
|
const progress = status.progress || { percent: '0' };
|
|
182
|
-
const aiStatusDetail = status.summaryStatus
|
|
203
|
+
const aiStatusDetail = status.summaryStatus ? ` [RegressionBot Summary: ${status.summaryStatus}]` : '';
|
|
183
204
|
process.stdout.write(`\r Status: ${status.status} (${progress.percent}%)${aiStatusDetail}`);
|
|
184
205
|
}, { waitForSummaries: !options['skip-summaries'] });
|
|
185
206
|
console.log('\n\n✅ Job Completed.');
|
|
@@ -197,12 +218,7 @@ Waiting for completion...
|
|
|
197
218
|
}
|
|
198
219
|
if (summary.regressionCount > 0) {
|
|
199
220
|
console.log('\n❌ Regressions found:');
|
|
200
|
-
summary.regressions.forEach(
|
|
201
|
-
console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
|
|
202
|
-
console.log(` Diff: ${r.diffUrl}`);
|
|
203
|
-
if (r.regressionbotSummary)
|
|
204
|
-
console.log(` Summary: ${r.regressionbotSummary}`);
|
|
205
|
-
});
|
|
221
|
+
summary.regressions.forEach(printRegression);
|
|
206
222
|
console.log(`\nTo approve these changes, run:\n npx regressionbot approve ${job.jobId}`);
|
|
207
223
|
process.exit(1);
|
|
208
224
|
}
|
|
@@ -237,12 +253,12 @@ Errors: ${summary.errorCount}
|
|
|
237
253
|
`);
|
|
238
254
|
if (summary.regressionCount > 0) {
|
|
239
255
|
console.log('❌ Regressions found:');
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
}
|
|
256
|
+
summary.regressions.forEach(printRegression);
|
|
257
|
+
}
|
|
258
|
+
if (summary.intentAssessment) {
|
|
259
|
+
const ia = summary.intentAssessment;
|
|
260
|
+
console.log(`\n🧭 Intent: ${ia.summary}`);
|
|
261
|
+
console.log(` bugs: ${ia.bugCount}, intentional: ${ia.intentionalCount}, noise: ${ia.noiseCount}, needs review: ${ia.needsReviewCount}`);
|
|
246
262
|
}
|
|
247
263
|
const doDownload = options.download || options['download-full'];
|
|
248
264
|
if (doDownload) {
|
package/dist/index.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary } from './types';
|
|
1
|
+
import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult } from './types';
|
|
2
2
|
import { sanitizeFilename, sanitizeUrlToPath } from './security';
|
|
3
3
|
export { sanitizeFilename, sanitizeUrlToPath, Viewports };
|
|
4
|
-
export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary };
|
|
4
|
+
export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult };
|
|
5
5
|
export declare class RegressionBot {
|
|
6
6
|
private apiKey;
|
|
7
7
|
private apiUrl;
|
|
@@ -40,13 +40,20 @@ export declare class RegressionBot {
|
|
|
40
40
|
options?: any;
|
|
41
41
|
}>;
|
|
42
42
|
masks?: string[];
|
|
43
|
+
/** CSS injected before screenshotting. Max 4096 characters. */
|
|
44
|
+
customCss?: string;
|
|
43
45
|
concurrency?: number;
|
|
44
46
|
autoApprove?: boolean;
|
|
47
|
+
/** What this run is testing, used to judge whether changes were intended. */
|
|
48
|
+
runContext?: RunContext;
|
|
45
49
|
}): Promise<JobHandle>;
|
|
46
50
|
/**
|
|
47
51
|
* Update the configuration settings for a named project.
|
|
52
|
+
*
|
|
53
|
+
* Changing anything that decides what a capture looks like invalidates the
|
|
54
|
+
* stored baselines — see {@link ProjectConfigUpdate}.
|
|
48
55
|
*/
|
|
49
|
-
updateProject(projectName: string, config:
|
|
56
|
+
updateProject(projectName: string, config: ProjectConfigUpdate): Promise<ProjectConfig>;
|
|
50
57
|
_request<T>(path: string, method?: string, body?: any): Promise<T>;
|
|
51
58
|
}
|
|
52
59
|
export declare class JobBuilder {
|
|
@@ -74,6 +81,17 @@ export declare class JobBuilder {
|
|
|
74
81
|
concurrency(n: number): this;
|
|
75
82
|
autoApprove(val?: boolean): this;
|
|
76
83
|
mask(selectors: string[]): this;
|
|
84
|
+
/**
|
|
85
|
+
* Inject custom CSS before each screenshot, e.g. to hide a dynamic widget:
|
|
86
|
+
* `'#chat-widget { display: none !important; }'`. Max 4096 characters.
|
|
87
|
+
*/
|
|
88
|
+
customCss(css: string): this;
|
|
89
|
+
/**
|
|
90
|
+
* Describe what this run is testing — commit, PR, expected changes — so
|
|
91
|
+
* RegressionBot can judge whether each change was intentional.
|
|
92
|
+
* Merges with anything set by an earlier call.
|
|
93
|
+
*/
|
|
94
|
+
withContext(context: RunContext): this;
|
|
77
95
|
/**
|
|
78
96
|
* Compiles the manifest and triggers the API.
|
|
79
97
|
*/
|
|
@@ -85,12 +103,7 @@ export declare class JobHandle {
|
|
|
85
103
|
constructor(sdk: RegressionBot, jobId: string);
|
|
86
104
|
getStatus(): Promise<JobStatus>;
|
|
87
105
|
getSummary(): Promise<JobSummary>;
|
|
88
|
-
approve(): Promise<
|
|
89
|
-
message: string;
|
|
90
|
-
jobId: string;
|
|
91
|
-
approvedUrlsCount: number;
|
|
92
|
-
failedCount?: number;
|
|
93
|
-
}>;
|
|
106
|
+
approve(): Promise<ApproveResult>;
|
|
94
107
|
/**
|
|
95
108
|
* Generate AI summaries on-demand for all regressions in a completed job.
|
|
96
109
|
*/
|
package/dist/index.js
CHANGED
|
@@ -55,9 +55,14 @@ class RegressionBot {
|
|
|
55
55
|
}
|
|
56
56
|
/**
|
|
57
57
|
* Update the configuration settings for a named project.
|
|
58
|
+
*
|
|
59
|
+
* Changing anything that decides what a capture looks like invalidates the
|
|
60
|
+
* stored baselines — see {@link ProjectConfigUpdate}.
|
|
58
61
|
*/
|
|
59
62
|
async updateProject(projectName, config) {
|
|
60
|
-
|
|
63
|
+
// The API wraps the updated project in { message, project }.
|
|
64
|
+
const res = await this._request(`/project/${encodeURIComponent(projectName)}`, 'PUT', config);
|
|
65
|
+
return res.project;
|
|
61
66
|
}
|
|
62
67
|
// Internal fetch wrapper
|
|
63
68
|
async _request(path, method = 'GET', body) {
|
|
@@ -140,6 +145,23 @@ class JobBuilder {
|
|
|
140
145
|
this.manifest.masks = selectors;
|
|
141
146
|
return this;
|
|
142
147
|
}
|
|
148
|
+
/**
|
|
149
|
+
* Inject custom CSS before each screenshot, e.g. to hide a dynamic widget:
|
|
150
|
+
* `'#chat-widget { display: none !important; }'`. Max 4096 characters.
|
|
151
|
+
*/
|
|
152
|
+
customCss(css) {
|
|
153
|
+
this.manifest.customCss = css;
|
|
154
|
+
return this;
|
|
155
|
+
}
|
|
156
|
+
/**
|
|
157
|
+
* Describe what this run is testing — commit, PR, expected changes — so
|
|
158
|
+
* RegressionBot can judge whether each change was intentional.
|
|
159
|
+
* Merges with anything set by an earlier call.
|
|
160
|
+
*/
|
|
161
|
+
withContext(context) {
|
|
162
|
+
this.manifest.runContext = { ...this.manifest.runContext, ...context };
|
|
163
|
+
return this;
|
|
164
|
+
}
|
|
143
165
|
/**
|
|
144
166
|
* Compiles the manifest and triggers the API.
|
|
145
167
|
*/
|
|
@@ -161,7 +183,9 @@ class JobBuilder {
|
|
|
161
183
|
scans: this.manifest.scans,
|
|
162
184
|
concurrency: this.manifest.concurrency,
|
|
163
185
|
autoApprove: this.manifest.autoApprove,
|
|
164
|
-
masks: this.manifest.masks
|
|
186
|
+
masks: this.manifest.masks,
|
|
187
|
+
customCss: this.manifest.customCss,
|
|
188
|
+
runContext: this.manifest.runContext
|
|
165
189
|
};
|
|
166
190
|
const res = await this.sdk._request('/crawl', 'POST', payload);
|
|
167
191
|
return new JobHandle(this.sdk, res.jobId);
|
|
@@ -203,22 +227,29 @@ class JobHandle {
|
|
|
203
227
|
const jobDir = path.join(baseDir, safeJobId);
|
|
204
228
|
if (!fs.existsSync(jobDir))
|
|
205
229
|
fs.mkdirSync(jobDir, { recursive: true });
|
|
206
|
-
const download = async (url,
|
|
230
|
+
const download = async (url, baseName) => {
|
|
207
231
|
try {
|
|
208
232
|
// 🛡️ SECURITY: Prevent SSRF and local file reads by enforcing HTTP(S) protocol
|
|
209
233
|
(0, security_1.validateProtocol)(url, 'download URL');
|
|
210
234
|
// 🛡️ SECURITY: Fetch with timeout
|
|
211
235
|
const res = await (0, security_1.fetchWithTimeout)(url);
|
|
212
236
|
if (!res.ok) {
|
|
213
|
-
console.warn(`Warning: Failed to download ${
|
|
237
|
+
console.warn(`Warning: Failed to download ${baseName} from ${url} (Status: ${res.status})`);
|
|
214
238
|
return;
|
|
215
239
|
}
|
|
240
|
+
// Determine file extension from Content-Type header
|
|
241
|
+
const contentType = res.headers.get('content-type') || '';
|
|
242
|
+
let ext = '.png'; // default fallback
|
|
243
|
+
if (contentType.includes('image/jpeg') || contentType.includes('image/jpg')) {
|
|
244
|
+
ext = '.jpg';
|
|
245
|
+
}
|
|
246
|
+
const finalName = `${baseName}${ext}`;
|
|
216
247
|
const buffer = Buffer.from(await res.arrayBuffer());
|
|
217
|
-
const filePath = path.join(jobDir,
|
|
248
|
+
const filePath = path.join(jobDir, finalName);
|
|
218
249
|
fs.writeFileSync(filePath, buffer);
|
|
219
250
|
}
|
|
220
251
|
catch (err) {
|
|
221
|
-
console.warn(`Warning: Failed to download ${
|
|
252
|
+
console.warn(`Warning: Failed to download ${baseName}: ${err.message}`);
|
|
222
253
|
}
|
|
223
254
|
};
|
|
224
255
|
// Regressions
|
|
@@ -226,12 +257,12 @@ class JobHandle {
|
|
|
226
257
|
const nameBase = (0, security_1.sanitizeUrlToPath)(r.url);
|
|
227
258
|
const safeVariant = (0, security_1.sanitizeFilename)(r.variantName);
|
|
228
259
|
if (r.diffUrl)
|
|
229
|
-
await download(r.diffUrl, `${nameBase}_diff_${safeVariant}
|
|
260
|
+
await download(r.diffUrl, `${nameBase}_diff_${safeVariant}`);
|
|
230
261
|
if (options.full) {
|
|
231
262
|
if (r.baselineUrl)
|
|
232
|
-
await download(r.baselineUrl, `${nameBase}_baseline_${safeVariant}
|
|
263
|
+
await download(r.baselineUrl, `${nameBase}_baseline_${safeVariant}`);
|
|
233
264
|
if (r.currentUrl)
|
|
234
|
-
await download(r.currentUrl, `${nameBase}_current_${safeVariant}
|
|
265
|
+
await download(r.currentUrl, `${nameBase}_current_${safeVariant}`);
|
|
235
266
|
}
|
|
236
267
|
}
|
|
237
268
|
// Matches (Full only)
|
|
@@ -240,9 +271,9 @@ class JobHandle {
|
|
|
240
271
|
const nameBase = (0, security_1.sanitizeUrlToPath)(m.url);
|
|
241
272
|
const safeVariant = (0, security_1.sanitizeFilename)(m.variantName);
|
|
242
273
|
if (m.baselineUrl)
|
|
243
|
-
await download(m.baselineUrl, `${nameBase}_baseline_${safeVariant}
|
|
274
|
+
await download(m.baselineUrl, `${nameBase}_baseline_${safeVariant}`);
|
|
244
275
|
if (m.currentUrl)
|
|
245
|
-
await download(m.currentUrl, `${nameBase}_current_${safeVariant}
|
|
276
|
+
await download(m.currentUrl, `${nameBase}_current_${safeVariant}`);
|
|
246
277
|
}
|
|
247
278
|
}
|
|
248
279
|
}
|
package/dist/types.d.ts
CHANGED
|
@@ -9,23 +9,137 @@ export interface ProjectScan {
|
|
|
9
9
|
exclude?: string[];
|
|
10
10
|
};
|
|
11
11
|
}
|
|
12
|
+
/**
|
|
13
|
+
* Credentials for reaching an origin behind an environment gate — basic auth,
|
|
14
|
+
* a bypass header, or a session cookie. Write-only: the API never returns these,
|
|
15
|
+
* only a `{ configured: true }` presence flag.
|
|
16
|
+
*/
|
|
17
|
+
export interface EnvGate {
|
|
18
|
+
basic?: {
|
|
19
|
+
username: string;
|
|
20
|
+
password: string;
|
|
21
|
+
};
|
|
22
|
+
headers?: Record<string, string>;
|
|
23
|
+
cookies?: Array<{
|
|
24
|
+
name: string;
|
|
25
|
+
value: string;
|
|
26
|
+
}>;
|
|
27
|
+
}
|
|
28
|
+
/** How a project's baseline advances between runs. Absent means `approved`. */
|
|
29
|
+
export type BaselinePolicy = 'approved' | 'rolling';
|
|
30
|
+
/** Frequency at which a project runs unattended. */
|
|
31
|
+
export type ProjectSchedule = 'hourly' | 'daily' | 'weekly';
|
|
32
|
+
/**
|
|
33
|
+
* A saved project as the API returns it.
|
|
34
|
+
*
|
|
35
|
+
* This is a read shape. To change a project, pass {@link ProjectConfigUpdate}
|
|
36
|
+
* to updateProject() — the two differ, because gate credentials are written as
|
|
37
|
+
* an {@link EnvGate} but read back only as a presence flag.
|
|
38
|
+
*/
|
|
12
39
|
export interface ProjectConfig {
|
|
13
|
-
orgId?: string;
|
|
14
40
|
name: string;
|
|
15
|
-
testOrigin
|
|
41
|
+
testOrigin: string;
|
|
16
42
|
baseOrigin?: string;
|
|
17
43
|
sitemapUrl?: string;
|
|
18
44
|
paths?: ProjectPath[];
|
|
19
45
|
scans?: ProjectScan[];
|
|
20
|
-
devices
|
|
46
|
+
devices: string[];
|
|
21
47
|
masks?: string[];
|
|
48
|
+
/** CSS injected before screenshotting, to hide dynamic widgets. Max 4096 characters. */
|
|
49
|
+
customCss?: string;
|
|
50
|
+
/** Presence flag only — the stored credential is never returned. */
|
|
51
|
+
testAuth?: {
|
|
52
|
+
configured: true;
|
|
53
|
+
};
|
|
54
|
+
/** Presence flag only — the stored credential is never returned. */
|
|
55
|
+
baseAuth?: {
|
|
56
|
+
configured: true;
|
|
57
|
+
};
|
|
22
58
|
concurrency?: number;
|
|
23
|
-
|
|
24
|
-
|
|
59
|
+
/** Extra instructions handed to the model that writes change summaries. */
|
|
60
|
+
aiPromptInstructions?: string;
|
|
61
|
+
/** Diff percentage below which a regression is skipped by the AI pass. Defaults to 0.01. */
|
|
62
|
+
aiSummaryThreshold?: number;
|
|
63
|
+
/**
|
|
64
|
+
* `approved` (default): a person accepts a run and its captures become the baseline.
|
|
65
|
+
* `rolling`: every completed managed run advances it, and those runs auto-approve.
|
|
66
|
+
*/
|
|
67
|
+
baselinePolicy?: BaselinePolicy;
|
|
68
|
+
/** Set when the project runs unattended. Absent means it only runs when triggered. */
|
|
69
|
+
schedule?: ProjectSchedule;
|
|
70
|
+
/**
|
|
71
|
+
* The UTC hour, 0–23, at which a `daily` or `weekly` project runs — `3` is 03:00 UTC.
|
|
72
|
+
* Absent means the slot anchors to whenever the first sweep picked the project up.
|
|
73
|
+
*/
|
|
74
|
+
scheduleHourUtc?: number;
|
|
75
|
+
/**
|
|
76
|
+
* The API key recorded when the schedule was set; every scheduled run is attributed
|
|
77
|
+
* to it. Absent if the schedule was set without an API key, in which case AI
|
|
78
|
+
* summaries are skipped for those runs.
|
|
79
|
+
*/
|
|
80
|
+
scheduleKeyId?: string;
|
|
81
|
+
/** When the scheduler last started a run. The next run is due one interval after this. */
|
|
82
|
+
lastScheduledRunAt?: string;
|
|
83
|
+
createdAt: string;
|
|
84
|
+
updatedAt: string;
|
|
25
85
|
lastRunAt?: string;
|
|
26
86
|
lastJobId?: string;
|
|
87
|
+
baselineVersion?: string;
|
|
27
88
|
baselineInvalidatedAt?: string;
|
|
28
89
|
}
|
|
90
|
+
/**
|
|
91
|
+
* The fields updateProject() accepts.
|
|
92
|
+
*
|
|
93
|
+
* Changing anything that decides what a capture looks like — testOrigin,
|
|
94
|
+
* baseOrigin, sitemapUrl, paths, scans, devices, masks, customCss, or a gate
|
|
95
|
+
* credential — invalidates the stored baselines, and the next run re-captures
|
|
96
|
+
* them instead of comparing. Everything else leaves baselines alone.
|
|
97
|
+
* Re-sending an identical value is not a change.
|
|
98
|
+
*/
|
|
99
|
+
export interface ProjectConfigUpdate {
|
|
100
|
+
testOrigin?: string;
|
|
101
|
+
baseOrigin?: string;
|
|
102
|
+
sitemapUrl?: string;
|
|
103
|
+
paths?: ProjectPath[];
|
|
104
|
+
scans?: ProjectScan[];
|
|
105
|
+
devices?: string[];
|
|
106
|
+
masks?: string[];
|
|
107
|
+
/** Max 4096 characters. */
|
|
108
|
+
customCss?: string;
|
|
109
|
+
/** Credentials for the test origin's environment gate. Pass null to clear. */
|
|
110
|
+
testAuth?: EnvGate | null;
|
|
111
|
+
/** Credentials for the base origin's environment gate. Pass null to clear. */
|
|
112
|
+
baseAuth?: EnvGate | null;
|
|
113
|
+
concurrency?: number;
|
|
114
|
+
aiPromptInstructions?: string;
|
|
115
|
+
aiSummaryThreshold?: number;
|
|
116
|
+
baselinePolicy?: BaselinePolicy;
|
|
117
|
+
/**
|
|
118
|
+
* Run this project unattended at this frequency. Requires `baselinePolicy: 'rolling'`
|
|
119
|
+
* on a managed project — the API rejects the combination without it, because on the
|
|
120
|
+
* approved policy an unapproved change is re-reported on every subsequent run.
|
|
121
|
+
* Live-vs-live projects store no baseline and are exempt. Pass null to remove.
|
|
122
|
+
*
|
|
123
|
+
* Without `scheduleHourUtc` the first run starts at the next hourly sweep and the
|
|
124
|
+
* cadence anchors to it.
|
|
125
|
+
*/
|
|
126
|
+
schedule?: ProjectSchedule | null;
|
|
127
|
+
/**
|
|
128
|
+
* Pin a `daily` or `weekly` schedule to this UTC hour, 0–23 — `3` means 03:00 UTC.
|
|
129
|
+
* The first run waits for the hour too, rather than firing immediately and settling
|
|
130
|
+
* into the hour from the following day. Hours only, because the scheduler sweeps once
|
|
131
|
+
* an hour; UTC only, because RegressionBot has no timezone to convert from.
|
|
132
|
+
*
|
|
133
|
+
* Rejected on an `hourly` schedule, which already runs every hour, and rejected if the
|
|
134
|
+
* project has no schedule to apply it to — either one already set, or one sent in the
|
|
135
|
+
* same update. Clearing the schedule with
|
|
136
|
+
* `schedule: null` clears this too, so re-enabling later cannot resurrect an hour
|
|
137
|
+
* nobody asked for on that run. Pass null to clear it on its own.
|
|
138
|
+
*
|
|
139
|
+
* A missed slot waits for the next one rather than catching up.
|
|
140
|
+
*/
|
|
141
|
+
scheduleHourUtc?: number | null;
|
|
142
|
+
}
|
|
29
143
|
export interface VRConfig {
|
|
30
144
|
apiKey?: string;
|
|
31
145
|
apiUrl?: string;
|
|
@@ -52,57 +166,105 @@ export declare const Viewports: {
|
|
|
52
166
|
readonly height: 667;
|
|
53
167
|
};
|
|
54
168
|
};
|
|
169
|
+
export type VerdictDecision = 'intentional' | 'bug' | 'noise' | 'needs_review';
|
|
170
|
+
/** How one labelled region of a diff was judged against the run's stated intent. */
|
|
171
|
+
export interface RegionVerdict {
|
|
172
|
+
decision: VerdictDecision;
|
|
173
|
+
/** 0–1. */
|
|
174
|
+
confidence: number;
|
|
175
|
+
reasoning: string;
|
|
176
|
+
}
|
|
177
|
+
/** The rolled-up verdict for one result, plus the per-region verdicts behind it. */
|
|
178
|
+
export interface ResultVerdict {
|
|
179
|
+
decision: VerdictDecision;
|
|
180
|
+
minConfidence: number;
|
|
181
|
+
avgConfidence: number;
|
|
182
|
+
/** Keyed by region label (A, B, C…). Empty when the verdict came from the text-only pass. */
|
|
183
|
+
regions: Record<string, RegionVerdict>;
|
|
184
|
+
}
|
|
55
185
|
export interface RegressionbotSummaryItem {
|
|
186
|
+
/** Region letter (A, B, C…), or an empty string for a whole-page change. */
|
|
56
187
|
label: string;
|
|
188
|
+
/** Single-sentence description of what changed in that region. */
|
|
57
189
|
text: string;
|
|
190
|
+
verdict?: RegionVerdict;
|
|
58
191
|
}
|
|
59
192
|
/**
|
|
60
193
|
* Result for a single URL + device comparison.
|
|
61
194
|
*
|
|
62
|
-
* Returned by
|
|
63
|
-
* (
|
|
64
|
-
*
|
|
195
|
+
* Returned by getStatus() and, split across `regressions`/`matches`, by
|
|
196
|
+
* getSummary(). Both endpoints pre-sign the image URLs; a URL is null when
|
|
197
|
+
* that image does not exist for this result (no baseline yet, no diff, …).
|
|
65
198
|
*/
|
|
66
199
|
export interface PageResult {
|
|
67
200
|
url: string;
|
|
68
|
-
/**
|
|
201
|
+
/** Whether the capture and comparison succeeded. */
|
|
202
|
+
status: 'SUCCESS' | 'ERROR';
|
|
203
|
+
/** Device variant tested (e.g. "Desktop Chrome", "iPhone 12"). */
|
|
69
204
|
variantName: string;
|
|
70
205
|
/** Percentage of pixels that differed from the baseline (0 = identical). */
|
|
71
206
|
diffPercentage: number;
|
|
72
|
-
/** Raw count of differing pixels. */
|
|
73
|
-
diffCount: number;
|
|
74
207
|
/** Perceptual similarity 0-100 (SSIM). */
|
|
75
208
|
visualMatchScore: number;
|
|
209
|
+
/** True when this capture became a baseline because none existed to compare against. */
|
|
210
|
+
isNewBaseline: boolean;
|
|
211
|
+
/** Why the capture or comparison failed. Present when status is ERROR. */
|
|
212
|
+
errorMessage?: string;
|
|
76
213
|
/** Plain-English description of what visually changed, generated by RegressionBot. */
|
|
77
214
|
regressionbotSummary?: RegressionbotSummaryItem[];
|
|
215
|
+
/**
|
|
216
|
+
* Whether the change was judged intentional, a bug, or noise. Only produced when
|
|
217
|
+
* the run carried a {@link RunContext}, and only once summaryStatus is COMPLETE.
|
|
218
|
+
*/
|
|
219
|
+
verdict?: ResultVerdict;
|
|
220
|
+
/** Selectors of the elements that changed, when the DOM comparison could identify them. */
|
|
221
|
+
elementsChanged?: string[];
|
|
222
|
+
/**
|
|
223
|
+
* Set when the DOM comparison could not run. The result is a plain pixel diff with
|
|
224
|
+
* no semantic change types, so treat its classifications as absent rather than as
|
|
225
|
+
* "nothing structural changed".
|
|
226
|
+
*/
|
|
227
|
+
domAssistSkipReason?: string;
|
|
78
228
|
/** Pre-signed URL for the stored baseline screenshot. */
|
|
79
|
-
baselineUrl
|
|
229
|
+
baselineUrl: string | null;
|
|
80
230
|
/** Pre-signed URL for the newly captured screenshot. */
|
|
81
|
-
currentUrl
|
|
231
|
+
currentUrl: string | null;
|
|
82
232
|
/** Pre-signed URL for the side-by-side annotated diff image. */
|
|
83
|
-
diffUrl
|
|
233
|
+
diffUrl: string | null;
|
|
234
|
+
/**
|
|
235
|
+
* Pre-signed URL for the diff mask. getStatus() always sends the key, null when
|
|
236
|
+
* there is no mask; getSummary() omits it entirely in that case.
|
|
237
|
+
*/
|
|
238
|
+
maskUrl?: string | null;
|
|
84
239
|
}
|
|
85
240
|
export interface JobProgress {
|
|
86
241
|
total: number;
|
|
87
242
|
completed: number;
|
|
88
243
|
percent: string;
|
|
89
244
|
}
|
|
245
|
+
/** Progress of the asynchronous AI summary pass. */
|
|
246
|
+
export type SummaryStatus = 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'FAILED';
|
|
90
247
|
export interface JobStatus {
|
|
91
248
|
jobId: string;
|
|
92
249
|
status: 'INITIALIZING' | 'PROCESSING' | 'FINISHING' | 'SUMMARIZING' | 'COMPLETED' | 'APPROVED' | 'FAILED';
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
250
|
+
/**
|
|
251
|
+
* PENDING means not yet started, COMPLETE that regressionbotSummary is populated
|
|
252
|
+
* (or that nothing needed one). Poll until COMPLETE before reading summaries.
|
|
253
|
+
*/
|
|
254
|
+
summaryStatus: SummaryStatus;
|
|
255
|
+
error: string | null;
|
|
256
|
+
progress: JobProgress;
|
|
257
|
+
executionTime: number;
|
|
258
|
+
createdAt: string;
|
|
259
|
+
/** Partial or complete results. Fills in as workers finish. */
|
|
260
|
+
results: PageResult[];
|
|
100
261
|
}
|
|
101
262
|
export interface JobSummary {
|
|
102
263
|
jobId: string;
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
264
|
+
/** getSummary() throws until the job reaches one of these. */
|
|
265
|
+
status: 'COMPLETED' | 'APPROVED' | 'FAILED';
|
|
266
|
+
summaryStatus: SummaryStatus;
|
|
267
|
+
error: string | null;
|
|
106
268
|
totalUrls: number;
|
|
107
269
|
completedCount: number;
|
|
108
270
|
/** Overall quality score 0–100 across all tested pages. */
|
|
@@ -112,9 +274,9 @@ export interface JobSummary {
|
|
|
112
274
|
matchCount: number;
|
|
113
275
|
newBaselineCount: number;
|
|
114
276
|
errorCount: number;
|
|
115
|
-
/** Pages where a visual regression was detected.
|
|
277
|
+
/** Pages where a visual regression was detected. */
|
|
116
278
|
regressions: PageResult[];
|
|
117
|
-
/** Pages that matched the baseline.
|
|
279
|
+
/** Pages that matched the baseline. */
|
|
118
280
|
matches: PageResult[];
|
|
119
281
|
/** Pages captured for the first time — no baseline existed to compare against. */
|
|
120
282
|
newBaselines: Array<{
|
|
@@ -127,6 +289,49 @@ export interface JobSummary {
|
|
|
127
289
|
variantName: string;
|
|
128
290
|
errorMessage: string;
|
|
129
291
|
}>;
|
|
292
|
+
/** Whole-job roll-up of how the changes line up with the run's stated intent. */
|
|
293
|
+
intentAssessment: IntentAssessment;
|
|
294
|
+
/** The intent this run was given, echoed back. */
|
|
295
|
+
runContext?: RunContext;
|
|
296
|
+
}
|
|
297
|
+
/**
|
|
298
|
+
* What a run is testing. Supplying it lets RegressionBot judge each change as
|
|
299
|
+
* intentional, a bug, or noise, rather than just reporting that pixels moved.
|
|
300
|
+
*/
|
|
301
|
+
export interface RunContext {
|
|
302
|
+
/** Short description of what this run is testing. */
|
|
303
|
+
changeDescription?: string;
|
|
304
|
+
gitCommitSha?: string;
|
|
305
|
+
/** Git commit subject line. */
|
|
306
|
+
gitCommitMessage?: string;
|
|
307
|
+
prTitle?: string;
|
|
308
|
+
/** Markdown. Truncated to 2000 characters by the API. */
|
|
309
|
+
prDescription?: string;
|
|
310
|
+
/** Explicit list of expected visual changes. */
|
|
311
|
+
expectedChanges?: string[];
|
|
312
|
+
/** Files or component names changed. */
|
|
313
|
+
scope?: string[];
|
|
314
|
+
}
|
|
315
|
+
export interface IntentAssessment {
|
|
316
|
+
/** False when the run carried no RunContext, in which case the counts are all zero. */
|
|
317
|
+
intentProvided: boolean;
|
|
318
|
+
bugCount: number;
|
|
319
|
+
intentionalCount: number;
|
|
320
|
+
noiseCount: number;
|
|
321
|
+
needsReviewCount: number;
|
|
322
|
+
/** True when every regression is accounted for by the stated intent. */
|
|
323
|
+
allAccountedFor: boolean;
|
|
324
|
+
/** One line summarising the job, safe to print straight into CI output. */
|
|
325
|
+
summary: string;
|
|
326
|
+
}
|
|
327
|
+
export interface ApproveResult {
|
|
328
|
+
message: string;
|
|
329
|
+
jobId: string;
|
|
330
|
+
approvedUrlsCount: number;
|
|
331
|
+
/** Present when some captures could not be promoted. */
|
|
332
|
+
failedCount?: number;
|
|
333
|
+
/** Pages skipped because another job updated their baseline first. */
|
|
334
|
+
conflictedUrls?: string[];
|
|
130
335
|
}
|
|
131
336
|
export interface JobAiSummary {
|
|
132
337
|
message: string;
|
package/package.json
CHANGED