@regressionbot/sdk 1.2.3 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +291 -8
- package/dist/cli.d.ts +27 -1
- package/dist/cli.js +194 -43
- package/dist/index.d.ts +27 -9
- package/dist/index.js +54 -15
- package/dist/types.d.ts +330 -27
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -13,7 +13,7 @@ RegressionBot is a declarative visual regression testing platform that helps you
|
|
|
13
13
|
|
|
14
14
|
Unlike traditional visual diffing libraries, RegressionBot is designed for modern, automated development loops and agentic pipelines:
|
|
15
15
|
|
|
16
|
-
- **Highly Accurate Regressions (
|
|
16
|
+
- **Highly Accurate Regressions (Less Noise)**: Leveraging advanced pixel-matching algorithms and element masking (using CSS selectors or the automatic `data-vr-mask` attribute), RegressionBot minimizes false positives caused by dynamic data, layout shifting, or third-party widgets.
|
|
17
17
|
- **Plain-English Summaries**: No more manual screenshot comparisons. RegressionBot translates visual diffs into concise, plain-English descriptions of what changed, so you know exactly what was modified at a glance.
|
|
18
18
|
- **Agentic Workflow Ready**: Built from the ground up to support autonomous coding agents (like Gemini or Claude) and automated developer loops. Through standard API endpoints, CLI commands, and Model Context Protocol (MCP) integrations, agents can trigger tests, read plain-English results, and approve baseline changes programmatically without human intervention.
|
|
19
19
|
|
|
@@ -24,6 +24,8 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
|
|
|
24
24
|
- **Matrix Testing**: Test multiple devices and viewports in a single job.
|
|
25
25
|
- **Auto-Discovery**: Scan sitemaps with glob patterns and limits.
|
|
26
26
|
- **RegressionBot Summaries**: Plain-English change descriptions for every regression, generated on-demand via the API.
|
|
27
|
+
- **Intent-Aware Verdicts**: Describe what a run is meant to change, and every regression comes back judged as intentional, a bug, or noise.
|
|
28
|
+
- **Scheduled Checks**: Run a saved project unattended, hourly, daily, or weekly — and at a set UTC hour.
|
|
27
29
|
- **Project-Based Baselines**: Save and reuse test configurations; share visual history across environments.
|
|
28
30
|
- **Auto-Approval**: Automatically promote screenshots to baselines on jobs that pass your criteria.
|
|
29
31
|
- **Zero Infrastructure**: No browser maintenance or server provisioning — RegressionBot handles it all.
|
|
@@ -34,6 +36,84 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
|
|
|
34
36
|
npm install @regressionbot/sdk
|
|
35
37
|
```
|
|
36
38
|
|
|
39
|
+
## Migration from 1.x
|
|
40
|
+
|
|
41
|
+
2.0.0 re-types the SDK against what the API actually returns, taken from the
|
|
42
|
+
orchestrator's implementation rather than from a spec that had drifted in both
|
|
43
|
+
directions. Most of the breakage is caught by the compiler. One is not:
|
|
44
|
+
|
|
45
|
+
```typescript
|
|
46
|
+
// 1.x — updateProject() returned the raw { message, project } envelope,
|
|
47
|
+
// so every field read off it was undefined at runtime
|
|
48
|
+
const project = await rb.updateProject('site', { schedule: 'daily' });
|
|
49
|
+
project.project.schedule; // this is where the value actually was
|
|
50
|
+
|
|
51
|
+
// 2.0 — the envelope is unwrapped for you
|
|
52
|
+
const project = await rb.updateProject('site', { schedule: 'daily' });
|
|
53
|
+
project.schedule; // 'daily'
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
The rest surface as type errors:
|
|
57
|
+
|
|
58
|
+
- **`PageResult.diffCount` is gone.** No endpoint ever returned it — both
|
|
59
|
+
`getJobStatus` and `getJobSummary` strip it. `PageResult` gains the fields they
|
|
60
|
+
do return: `status`, `changed`, `isNewBaseline`, `errorMessage`,
|
|
61
|
+
`elementsChanged`, `domAssistSkipReason` and `verdict`. Image URLs are typed
|
|
62
|
+
nullable, which they always were.
|
|
63
|
+
- **`updateProject()` takes `ProjectConfigUpdate`, not `Partial<ProjectConfig>`.**
|
|
64
|
+
The read and write shapes genuinely differ: a gate credential is written as an
|
|
65
|
+
`EnvGate` and read back only as `{ configured: true }`.
|
|
66
|
+
- **`ProjectConfig.orgId` is gone** — the API stopped returning it.
|
|
67
|
+
- **`summaryStatus` can be `FAILED`**, which the old union omitted. `error`,
|
|
68
|
+
`progress`, `results`, `createdAt` and `intentAssessment` are always returned.
|
|
69
|
+
- **`approve()` gains `conflictedUrls`**, returned when another job updated a
|
|
70
|
+
baseline first.
|
|
71
|
+
|
|
72
|
+
New in 2.0 and requiring a current API: `.customCss()` and `.withContext()` on
|
|
73
|
+
the builder, intent-aware verdicts, environment gates, baseline policies, and
|
|
74
|
+
scheduling. `scheduleHourUtc` needs API 2.7.0 or later — on an older API the
|
|
75
|
+
field is accepted and ignored, and the schedule stays anchored to its first run.
|
|
76
|
+
|
|
77
|
+
### 2.0.1 — never published separately; included in 2.1.0
|
|
78
|
+
|
|
79
|
+
Upgrading from 2.0.0 goes straight to 2.1.0, so read this section as well as the
|
|
80
|
+
next one.
|
|
81
|
+
|
|
82
|
+
- **The builder no longer invents defaults.** It used to send `concurrency: 10`
|
|
83
|
+
on every run, plus an empty `devices` list. The API compares every parameter it
|
|
84
|
+
receives against the saved project config and rejects the run on a mismatch, so
|
|
85
|
+
a project storing any other concurrency — including none, which is every project
|
|
86
|
+
created outside the SDK — failed with *"Params differ from stored config"* and
|
|
87
|
+
there was no way to unset the field. Call `.concurrency(n)` only when you mean
|
|
88
|
+
it; unset now means the API's default of 4, not 10.
|
|
89
|
+
- **CLI: `--concurrency` now rejects a missing or non-numeric value** instead of
|
|
90
|
+
coercing it. `--concurrency` with no value used to run at 1, and a
|
|
91
|
+
non-numeric one used to run at 10; both now fail with a message. Passing a
|
|
92
|
+
valid `1`–`20` is unchanged. Misuse exits **2**, so CI can tell a bad flag
|
|
93
|
+
from a run that worked and found regressions — both of which used to exit 1.
|
|
94
|
+
- **`PageResult.maskUrl` is gone.** It was never on the public contract: both
|
|
95
|
+
endpoints emit it to internal callers only, so code reading it was already
|
|
96
|
+
getting `undefined` and nothing changes at runtime. It was listed here in
|
|
97
|
+
2.0.0 by mistake. There is no public replacement — the mask is a transparent
|
|
98
|
+
overlay built for the dashboard, and `diffUrl` is the annotated diff image you
|
|
99
|
+
want instead.
|
|
100
|
+
- **`PageResult.changed` added**, with `contentChanged` alongside it. This is the
|
|
101
|
+
rule the API uses to split `regressions` from `matches`. Read it instead of
|
|
102
|
+
testing `diffPercentage === 0` — a text edit that moves no pixels is a
|
|
103
|
+
regression at 0.00%.
|
|
104
|
+
|
|
105
|
+
### 2.1.0
|
|
106
|
+
|
|
107
|
+
- **`--fail-on unintended`** on the CLI: exit 1 only on changes judged a bug or
|
|
108
|
+
needing review, so expected changes keep CI green. Comes with
|
|
109
|
+
`--change-description`, `--expected-changes`, `--pr-title` and `--commit` to
|
|
110
|
+
describe the run. Default behaviour is unchanged.
|
|
111
|
+
- **`--flag=value` is now accepted** for every flag. Required when the value starts
|
|
112
|
+
with `--`, such as a CSS custom property: `--custom-css="--brand: #fff"` used to
|
|
113
|
+
drop its value silently.
|
|
114
|
+
- The run command now reports capture errors even when there are also regressions.
|
|
115
|
+
They used to be hidden behind the regression branch.
|
|
116
|
+
|
|
37
117
|
## Usage
|
|
38
118
|
|
|
39
119
|
### Basic Example
|
|
@@ -84,13 +164,20 @@ const job = await rb
|
|
|
84
164
|
// Discovery: Auto-discover up to 20 blog posts
|
|
85
165
|
.scan('/blog/**', { limit: 20 })
|
|
86
166
|
|
|
87
|
-
// Concurrency:
|
|
88
|
-
.
|
|
167
|
+
// Concurrency: pages captured in parallel, 1-20. Omit it to take the default of 4 —
|
|
168
|
+
// the load lands on the site being captured, not on RegressionBot.
|
|
169
|
+
.concurrency(8)
|
|
89
170
|
|
|
90
171
|
// Masking: Automatic and manual masking
|
|
91
172
|
.mask(['.ads', '#modal']) // Manual selectors
|
|
92
173
|
// Tip: Adding 'data-vr-mask' to your HTML elements masks them automatically!
|
|
93
174
|
|
|
175
|
+
// Custom CSS: Injected before every screenshot (max 4096 chars)
|
|
176
|
+
.customCss('#chat-widget { display: none !important; }')
|
|
177
|
+
|
|
178
|
+
// Intent: What this run is testing, so changes can be judged against it
|
|
179
|
+
.withContext({ changeDescription: 'Restyle the pricing table' })
|
|
180
|
+
|
|
94
181
|
// Execute: Compiles manifest and triggers the API
|
|
95
182
|
.run();
|
|
96
183
|
|
|
@@ -112,9 +199,9 @@ const summary = await job.getSummary();
|
|
|
112
199
|
if (summary.regressions.length > 0) {
|
|
113
200
|
console.log(`\n${summary.regressions.length} regressions found:`);
|
|
114
201
|
for (const regression of summary.regressions) {
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
console.log(
|
|
202
|
+
for (const item of regression.regressionbotSummary ?? []) {
|
|
203
|
+
// `label` is the region letter (A, B, C…), or '' for a whole-page change
|
|
204
|
+
console.log(`[${item.label}] ${item.text}`);
|
|
118
205
|
}
|
|
119
206
|
}
|
|
120
207
|
}
|
|
@@ -124,6 +211,107 @@ const aiResult = await job.generateAiSummary();
|
|
|
124
211
|
console.log(`Generated summaries for ${aiResult.summaries.length} regressions.`);
|
|
125
212
|
```
|
|
126
213
|
|
|
214
|
+
### Reading exactly what changed
|
|
215
|
+
|
|
216
|
+
`changes` is computed by diffing the two documents, so unlike the prose summary it is
|
|
217
|
+
measured rather than described — you can quote it directly.
|
|
218
|
+
|
|
219
|
+
```typescript
|
|
220
|
+
for (const r of summary.regressions) {
|
|
221
|
+
for (const c of r.changes ?? []) {
|
|
222
|
+
switch (c.type) {
|
|
223
|
+
case 'text-edit':
|
|
224
|
+
console.log(`${c.element}: "${c.before}" → "${c.after}"`);
|
|
225
|
+
break;
|
|
226
|
+
case 'style-only':
|
|
227
|
+
for (const [prop, [from, to]] of Object.entries(c.style ?? {})) {
|
|
228
|
+
console.log(`${c.element}: ${prop} ${from} → ${to}`);
|
|
229
|
+
}
|
|
230
|
+
break;
|
|
231
|
+
default:
|
|
232
|
+
console.log(`${c.type} on ${c.element}`);
|
|
233
|
+
}
|
|
234
|
+
}
|
|
235
|
+
}
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
Two things to know before you rely on it:
|
|
239
|
+
|
|
240
|
+
- **It can be truncated.** At most 40 changes per page, and fewer if they would exceed the
|
|
241
|
+
API's 8 KB budget for the field. Treat it as the most significant changes, not a
|
|
242
|
+
guaranteed-complete list.
|
|
243
|
+
- **`changes` absent is not "nothing changed".** It means the document comparison could not
|
|
244
|
+
run on that page — check `domAssistSkipReason`.
|
|
245
|
+
|
|
246
|
+
`box` gives you the location in the capture, in CSS pixels from the top-left of the
|
|
247
|
+
full-page image, so it indexes straight into `currentUrl` if you want to crop:
|
|
248
|
+
|
|
249
|
+
```typescript
|
|
250
|
+
const aboveTheFold = r.changes?.filter(c => c.box && c.box.y < 800) ?? [];
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
### Intent-Aware Verdicts
|
|
254
|
+
|
|
255
|
+
Tell RegressionBot what a run is meant to change, and every regression comes back judged
|
|
256
|
+
against that intent — `intentional`, `bug`, `noise`, or `needs_review`.
|
|
257
|
+
|
|
258
|
+
```typescript
|
|
259
|
+
const job = await rb
|
|
260
|
+
.test(process.env.VERCEL_PREVIEW_URL)
|
|
261
|
+
.forProject('marketing-site-v2')
|
|
262
|
+
.withContext({
|
|
263
|
+
changeDescription: 'Restyle the pricing table',
|
|
264
|
+
gitCommitSha: process.env.GITHUB_SHA,
|
|
265
|
+
prTitle: process.env.PR_TITLE,
|
|
266
|
+
expectedChanges: ['Pricing table background is now dark'],
|
|
267
|
+
scope: ['components/PricingTable.tsx'],
|
|
268
|
+
})
|
|
269
|
+
.run();
|
|
270
|
+
|
|
271
|
+
await job.waitForCompletion(2000, undefined, { waitForSummaries: true });
|
|
272
|
+
const summary = await job.getSummary();
|
|
273
|
+
|
|
274
|
+
// Whole-job roll-up: does everything that changed line up with the stated intent?
|
|
275
|
+
console.log(summary.intentAssessment?.summary);
|
|
276
|
+
console.log(`Bugs: ${summary.intentAssessment?.bugCount}`);
|
|
277
|
+
|
|
278
|
+
// Per-result verdict, plus the per-region verdicts behind it
|
|
279
|
+
for (const r of summary.regressions) {
|
|
280
|
+
console.log(`${r.url}: ${r.verdict?.decision} (${r.verdict?.avgConfidence})`);
|
|
281
|
+
}
|
|
282
|
+
|
|
283
|
+
// Fail CI only on changes that were not intended
|
|
284
|
+
const bugs = summary.regressions.filter(r => r.verdict?.decision === 'bug');
|
|
285
|
+
if (bugs.length > 0) process.exit(1);
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
`runProject()` takes the same context, so a run from a saved project can be
|
|
289
|
+
intent-aware too:
|
|
290
|
+
|
|
291
|
+
```typescript
|
|
292
|
+
const job = await rb.runProject('marketing-site-v2', {
|
|
293
|
+
testOrigin: process.env.VERCEL_PREVIEW_URL,
|
|
294
|
+
runContext: { changeDescription: 'Restyle the pricing table' },
|
|
295
|
+
});
|
|
296
|
+
```
|
|
297
|
+
|
|
298
|
+
### Hiding Dynamic Content
|
|
299
|
+
|
|
300
|
+
`.mask()` hides elements by selector. `.customCss()` injects arbitrary CSS before each
|
|
301
|
+
screenshot when masking isn't enough — collapsing an animation, pinning a carousel, or
|
|
302
|
+
neutralising a third-party widget. Max 4096 characters.
|
|
303
|
+
|
|
304
|
+
```typescript
|
|
305
|
+
const job = await rb
|
|
306
|
+
.test('https://preview.myapp.com')
|
|
307
|
+
.forProject('my-app-web')
|
|
308
|
+
.customCss(`
|
|
309
|
+
#chat-widget { display: none !important; }
|
|
310
|
+
* { animation: none !important; transition: none !important; }
|
|
311
|
+
`)
|
|
312
|
+
.run();
|
|
313
|
+
```
|
|
314
|
+
|
|
127
315
|
### Progress Tracking
|
|
128
316
|
|
|
129
317
|
Pass a callback to `waitForCompletion` to receive status updates while the job runs:
|
|
@@ -177,6 +365,69 @@ const summary = await job.getSummary();
|
|
|
177
365
|
console.log(`Score: ${summary.overallScore}/100`);
|
|
178
366
|
```
|
|
179
367
|
|
|
368
|
+
### Environment Gates
|
|
369
|
+
|
|
370
|
+
If an origin sits behind basic auth, a bypass header, or a session cookie, store the
|
|
371
|
+
credential on the project. It is encrypted at rest and never returned — reads report
|
|
372
|
+
only `{ configured: true }`.
|
|
373
|
+
|
|
374
|
+
```typescript
|
|
375
|
+
await rb.updateProject('marketing-site-v2', {
|
|
376
|
+
testAuth: { basic: { username: 'preview', password: process.env.PREVIEW_PASSWORD! } },
|
|
377
|
+
// or: { headers: { 'x-vercel-protection-bypass': token } }
|
|
378
|
+
// or: { cookies: [{ name: 'session', value: token }] }
|
|
379
|
+
});
|
|
380
|
+
|
|
381
|
+
const project = await rb.getProject('marketing-site-v2');
|
|
382
|
+
console.log(project.testAuth); // { configured: true }
|
|
383
|
+
|
|
384
|
+
// Clear it
|
|
385
|
+
await rb.updateProject('marketing-site-v2', { testAuth: null });
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
Setting or clearing a gate credential invalidates the stored baselines, since it changes
|
|
389
|
+
what the capture can reach.
|
|
390
|
+
|
|
391
|
+
### Scheduled Checks
|
|
392
|
+
|
|
393
|
+
A project can run unattended on a fixed cadence. Scheduling requires
|
|
394
|
+
`baselinePolicy: 'rolling'` on a managed project — on the default `approved` policy an
|
|
395
|
+
unapproved change would be re-reported on every subsequent run, so the API rejects the
|
|
396
|
+
combination. Live-vs-live projects store no baseline and are exempt.
|
|
397
|
+
|
|
398
|
+
```typescript
|
|
399
|
+
// Turn on a daily unattended check
|
|
400
|
+
await rb.updateProject('marketing-site-v2', {
|
|
401
|
+
baselinePolicy: 'rolling',
|
|
402
|
+
schedule: 'daily', // 'hourly' | 'daily' | 'weekly'
|
|
403
|
+
});
|
|
404
|
+
|
|
405
|
+
// Or pin it to a UTC hour — this one runs at 03:00 UTC every day
|
|
406
|
+
await rb.updateProject('marketing-site-v2', {
|
|
407
|
+
baselinePolicy: 'rolling',
|
|
408
|
+
schedule: 'daily',
|
|
409
|
+
scheduleHourUtc: 3, // 0–23, UTC
|
|
410
|
+
});
|
|
411
|
+
|
|
412
|
+
// Turn it back off (this clears scheduleHourUtc too)
|
|
413
|
+
await rb.updateProject('marketing-site-v2', { schedule: null });
|
|
414
|
+
|
|
415
|
+
// Read back when the scheduler last fired — the next run is due one interval after this
|
|
416
|
+
const project = await rb.getProject('marketing-site-v2');
|
|
417
|
+
console.log(project.schedule, project.scheduleHourUtc, project.lastScheduledRunAt);
|
|
418
|
+
```
|
|
419
|
+
|
|
420
|
+
Without `scheduleHourUtc` the first run starts at the next hourly sweep and the cadence
|
|
421
|
+
anchors to it. With it, the first run waits for that hour as well. Hours only, since the
|
|
422
|
+
scheduler sweeps once an hour, and UTC only. It is rejected on an `hourly` schedule, and
|
|
423
|
+
rejected on a project with no schedule to apply it to. Clearing the schedule clears the
|
|
424
|
+
hour with it, and a missed slot waits for the next one rather than catching up.
|
|
425
|
+
|
|
426
|
+
Setting a schedule, its hour, or a baseline policy does **not** invalidate
|
|
427
|
+
baselines — only changes that affect what a capture looks like (`testOrigin`,
|
|
428
|
+
`baseOrigin`, `sitemapUrl`, `paths`, `scans`, `devices`, `masks`, `customCss`) do that.
|
|
429
|
+
Billing is per comparison, so cost scales with frequency.
|
|
430
|
+
|
|
180
431
|
### Reconnecting to an Existing Job
|
|
181
432
|
|
|
182
433
|
If you have a job ID from a previous run (e.g., stored in CI state), you can attach to it without re-running the test:
|
|
@@ -210,7 +461,39 @@ Test an entire site using glob patterns.
|
|
|
210
461
|
npx @regressionbot/sdk https://example.com --project my-project --scan "/**" --exclude "/admin/**" --concurrency 20
|
|
211
462
|
```
|
|
212
463
|
|
|
213
|
-
|
|
464
|
+
Use `--mask` to hide elements by selector, or `--custom-css` to inject CSS before every
|
|
465
|
+
screenshot:
|
|
466
|
+
```bash
|
|
467
|
+
npx @regressionbot/sdk https://example.com --project my-site --custom-css "#chat-widget { display: none !important; }"
|
|
468
|
+
```
|
|
469
|
+
|
|
470
|
+
Any flag also accepts `--flag=value`. You need that form when the value itself starts with
|
|
471
|
+
`--`, such as a CSS custom property — otherwise it looks like the next flag and is dropped:
|
|
472
|
+
```bash
|
|
473
|
+
npx @regressionbot/sdk https://example.com --project my-site --custom-css="--brand-bg: #fff"
|
|
474
|
+
```
|
|
475
|
+
|
|
476
|
+
#### 3. Failing CI only on unintended changes
|
|
477
|
+
|
|
478
|
+
By default any regression exits 1. Describe what the run is meant to change and
|
|
479
|
+
`--fail-on unintended` will keep the build green for changes that match, failing only on
|
|
480
|
+
what looks like a bug:
|
|
481
|
+
|
|
482
|
+
```bash
|
|
483
|
+
npx @regressionbot/sdk https://preview.example.com \
|
|
484
|
+
--project my-site \
|
|
485
|
+
--change-description "Restyle the pricing table" \
|
|
486
|
+
--expected-changes "Pricing table is now dark,Buttons are rounded" \
|
|
487
|
+
--commit "$GITHUB_SHA" \
|
|
488
|
+
--fail-on unintended
|
|
489
|
+
```
|
|
490
|
+
|
|
491
|
+
A change with no verdict always fails the build — if the run carried no intent, or the
|
|
492
|
+
summary pass hasn't finished, nothing judged it, and treating that as expected would turn a
|
|
493
|
+
missing verdict into a silent pass. So `--fail-on unintended` without
|
|
494
|
+
`--change-description` behaves like `--fail-on any`, and warns that it will.
|
|
495
|
+
|
|
496
|
+
#### 4. Job Summary
|
|
214
497
|
Get detailed results and diff URLs for a completed job.
|
|
215
498
|
```bash
|
|
216
499
|
npx @regressionbot/sdk summary <jobId>
|
|
@@ -226,7 +509,7 @@ Use the `--download-full` flag to save baseline, current, and diff images:
|
|
|
226
509
|
npx @regressionbot/sdk summary <jobId> --download-full
|
|
227
510
|
```
|
|
228
511
|
|
|
229
|
-
####
|
|
512
|
+
#### 5. Approve Changes
|
|
230
513
|
Promote the current screenshots of a job to be the new baselines.
|
|
231
514
|
```bash
|
|
232
515
|
npx @regressionbot/sdk approve <jobId>
|
package/dist/cli.d.ts
CHANGED
|
@@ -1,2 +1,28 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
|
|
2
|
+
import { PageResult, RunContext } from './types';
|
|
3
|
+
declare function printRegression(r: PageResult): void;
|
|
4
|
+
type FailOn = 'any' | 'unintended';
|
|
5
|
+
declare function parseFailOn(raw: unknown): FailOn;
|
|
6
|
+
/**
|
|
7
|
+
* Whether one regression should fail the build under `--fail-on unintended`.
|
|
8
|
+
*
|
|
9
|
+
* A regression with no verdict always blocks. Nothing judged it — the run carried no
|
|
10
|
+
* intent, or the summary pass had not finished — and treating unjudged as wanted would
|
|
11
|
+
* turn a missing verdict into a silent pass, which is the one outcome a regression
|
|
12
|
+
* detector must never produce.
|
|
13
|
+
*/
|
|
14
|
+
declare function isBlocking(r: PageResult): boolean;
|
|
15
|
+
/**
|
|
16
|
+
* Splits regressions into the ones that fail the build and a count of the ones excused.
|
|
17
|
+
*
|
|
18
|
+
* Pure, and exported, because this is the decision that sets the exit code — the whole
|
|
19
|
+
* point of --fail-on. Testing isBlocking alone proves nothing about whether the caller
|
|
20
|
+
* applies it correctly for each mode.
|
|
21
|
+
*/
|
|
22
|
+
declare function selectBlocking(regressions: PageResult[], failOn: FailOn): {
|
|
23
|
+
blocking: PageResult[];
|
|
24
|
+
excused: number;
|
|
25
|
+
};
|
|
26
|
+
declare function buildRunContext(options: any): RunContext | undefined;
|
|
27
|
+
declare function parseArgs(args: string[]): any;
|
|
28
|
+
export { parseArgs, parseFailOn, isBlocking, selectBlocking, buildRunContext, printRegression };
|