@regressionbot/sdk 2.3.0 → 2.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +66 -5
- package/dist/cli.js +2 -0
- package/dist/types.d.ts +109 -12
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -25,7 +25,7 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
|
|
|
25
25
|
- **Auto-Discovery**: Scan sitemaps with glob patterns and limits.
|
|
26
26
|
- **RegressionBot Summaries**: Plain-English change descriptions for every regression, generated on-demand via the API.
|
|
27
27
|
- **Intent-Aware Verdicts**: Describe what a run is meant to change, and every regression comes back judged as intentional, a bug, or noise.
|
|
28
|
-
- **Scheduled Checks**: Run a saved project unattended,
|
|
28
|
+
- **Scheduled Checks**: Run a saved project unattended, daily or weekly — and at a set UTC hour.
|
|
29
29
|
- **Project-Based Baselines**: Save and reuse test configurations; share visual history across environments.
|
|
30
30
|
- **Auto-Approval**: Automatically promote screenshots to baselines on jobs that pass your criteria.
|
|
31
31
|
- **Zero Infrastructure**: No browser maintenance or server provisioning — RegressionBot handles it all.
|
|
@@ -74,6 +74,28 @@ the builder, intent-aware verdicts, environment gates, baseline policies, and
|
|
|
74
74
|
scheduling. `scheduleHourUtc` needs API 2.7.0 or later — on an older API the
|
|
75
75
|
field is accepted and ignored, and the schedule stays anchored to its first run.
|
|
76
76
|
|
|
77
|
+
### 2.4.0
|
|
78
|
+
|
|
79
|
+
Needs API 2.9.0 or later. On an older API the new fields are simply absent.
|
|
80
|
+
|
|
81
|
+
- **`'hourly'` is gone from `ProjectSchedule`.** The API stopped accepting it on
|
|
82
|
+
2026-08-20 and rejects it outright, so code that sends it already fails at runtime;
|
|
83
|
+
now it fails at compile time instead. Use `'daily'` (with `scheduleHourUtc`) or
|
|
84
|
+
`'weekly'`. Kept as a minor release because no working call can depend on it.
|
|
85
|
+
- **Environment labels.** `environment` and `baseEnvironment` on a project
|
|
86
|
+
(`updateProject`, `getProject`) and on each run's status and summary, so a run reads
|
|
87
|
+
"staging vs production" instead of a bare URL. A label only: never resets baselines.
|
|
88
|
+
- **A standing `intent` on a project**, which scheduled runs are judged against.
|
|
89
|
+
- **`runSummary`** on the job summary: a short AI reading of the whole run.
|
|
90
|
+
- **`changeGroup`** on each regression (`structural`, `edited`, `metadata`,
|
|
91
|
+
`cosmetic`), the same grouping the digest email uses.
|
|
92
|
+
- **New change detail:** `attribute-edit` and `font-failed` change types, and `label`,
|
|
93
|
+
`attributes` and `valueOnly` on a change.
|
|
94
|
+
- **`project`, `origin`, `baseOrigin`, `totalUrls` and `completedCount`** on the job status.
|
|
95
|
+
- **Deprecated:** `visualMatchScore`, `overallScore`, `domAssistSkipReason`,
|
|
96
|
+
`elementsChanged` and a summary item's `verdict`. All still typed; none measure what
|
|
97
|
+
their names say under the DOM engine.
|
|
98
|
+
|
|
77
99
|
### 2.3.0
|
|
78
100
|
|
|
79
101
|
Needs API 2.9.0 or later. On an older API the verdict fields are simply absent, with
|
|
@@ -290,6 +312,16 @@ Two things to know before you rely on it:
|
|
|
290
312
|
- **`changes` absent is not "nothing changed".** It means the document comparison could not
|
|
291
313
|
run on that page — check `domAssistSkipReason`.
|
|
292
314
|
|
|
315
|
+
Each change also carries a `type` beyond the ones above — `attribute-edit` for a reader-facing
|
|
316
|
+
attribute (`href`, `alt`, `aria-label`, …; see `c.attributes`) and `font-failed` for a web font
|
|
317
|
+
that didn't load. A change with no words of its own (`style-only`, `attribute-edit`,
|
|
318
|
+
`image-change`) may carry a short `label` naming the element instead, and a date-only edit
|
|
319
|
+
("Sep 17" → "Sep 23") is reported with `valueOnly: 'date'` without counting as a change.
|
|
320
|
+
|
|
321
|
+
Every regression also carries a `changeGroup` — `structural`, `edited`, `metadata`,
|
|
322
|
+
`cosmetic`, or (on older results) `unmeasured`/`date` — the same grouping the digest email
|
|
323
|
+
uses, so you can filter a run down to one kind of change without re-deriving the rule.
|
|
324
|
+
|
|
293
325
|
`box` gives you the location in the capture, in CSS pixels from the top-left of the
|
|
294
326
|
full-page image, so it indexes straight into `currentUrl` if you want to crop:
|
|
295
327
|
|
|
@@ -477,7 +509,7 @@ combination. Live-vs-live projects store no baseline and are exempt.
|
|
|
477
509
|
// Turn on a daily unattended check
|
|
478
510
|
await rb.updateProject('marketing-site-v2', {
|
|
479
511
|
baselinePolicy: 'rolling',
|
|
480
|
-
schedule: 'daily', // '
|
|
512
|
+
schedule: 'daily', // 'daily' | 'weekly'
|
|
481
513
|
});
|
|
482
514
|
|
|
483
515
|
// Or pin it to a UTC hour — this one runs at 03:00 UTC every day
|
|
@@ -497,15 +529,44 @@ console.log(project.schedule, project.scheduleHourUtc, project.lastScheduledRunA
|
|
|
497
529
|
|
|
498
530
|
Without `scheduleHourUtc` the first run starts at the next hourly sweep and the cadence
|
|
499
531
|
anchors to it. With it, the first run waits for that hour as well. Hours only, since the
|
|
500
|
-
scheduler sweeps once an hour, and UTC only. It is rejected on
|
|
501
|
-
|
|
502
|
-
|
|
532
|
+
scheduler sweeps once an hour, and UTC only. It is rejected on a project with no schedule
|
|
533
|
+
to apply it to. Clearing the schedule clears the hour with it, and a missed slot waits
|
|
534
|
+
for the next one rather than catching up.
|
|
503
535
|
|
|
504
536
|
Setting a schedule, its hour, or a baseline policy does **not** invalidate
|
|
505
537
|
baselines — only changes that affect what a capture looks like (`testOrigin`,
|
|
506
538
|
`baseOrigin`, `sitemapUrl`, `paths`, `scans`, `devices`, `masks`, `customCss`) do that.
|
|
507
539
|
Billing is per comparison, so cost scales with frequency.
|
|
508
540
|
|
|
541
|
+
A scheduled run has no PR or commit to draw an intent from, so give the project a standing
|
|
542
|
+
`intent` instead — every scheduled run is judged against it:
|
|
543
|
+
|
|
544
|
+
```typescript
|
|
545
|
+
await rb.updateProject('marketing-site-v2', {
|
|
546
|
+
intent: 'Content is edited in the CMS daily; navigation, pricing and layout must not change.',
|
|
547
|
+
});
|
|
548
|
+
```
|
|
549
|
+
|
|
550
|
+
Without one, a scheduled run's `intentAssessment.decision` reads `not_judged`. Runs started
|
|
551
|
+
by a caller keep their own `runContext` and are never judged against the project's `intent`.
|
|
552
|
+
Pass `null` or an empty string to remove it. Does not invalidate baselines.
|
|
553
|
+
|
|
554
|
+
### Environment Labels
|
|
555
|
+
|
|
556
|
+
Label which environment a project's origins are — `environment` for `testOrigin`,
|
|
557
|
+
`baseEnvironment` for `baseOrigin` on a live-vs-live project — so runs, emails and the
|
|
558
|
+
dashboard read "staging vs production" instead of a bare URL:
|
|
559
|
+
|
|
560
|
+
```typescript
|
|
561
|
+
await rb.updateProject('marketing-site-v2', { environment: 'staging' });
|
|
562
|
+
|
|
563
|
+
const project = await rb.getProject('marketing-site-v2');
|
|
564
|
+
console.log(project.environment); // 'staging'
|
|
565
|
+
```
|
|
566
|
+
|
|
567
|
+
A label only: 1–24 letters, digits, spaces or hyphens, stored lower-case. It never resets
|
|
568
|
+
baselines and never changes a run. Pass `null` or an empty string to remove it.
|
|
569
|
+
|
|
509
570
|
### Reconnecting to an Existing Job
|
|
510
571
|
|
|
511
572
|
If you have a job ID from a previous run (e.g., stored in CI state), you can attach to it without re-running the test:
|
package/dist/cli.js
CHANGED
|
@@ -53,6 +53,8 @@ function formatSummary(items) {
|
|
|
53
53
|
}
|
|
54
54
|
function printRegression(r) {
|
|
55
55
|
console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
|
|
56
|
+
if (r.changeGroup)
|
|
57
|
+
console.log(` Group: ${r.changeGroup}`);
|
|
56
58
|
if (r.diffUrl)
|
|
57
59
|
console.log(` Diff: ${r.diffUrl}`);
|
|
58
60
|
if (r.verdict) {
|
package/dist/types.d.ts
CHANGED
|
@@ -28,7 +28,7 @@ export interface EnvGate {
|
|
|
28
28
|
/** How a project's baseline advances between runs. Absent means `approved`. */
|
|
29
29
|
export type BaselinePolicy = 'approved' | 'rolling';
|
|
30
30
|
/** Frequency at which a project runs unattended. */
|
|
31
|
-
export type ProjectSchedule = '
|
|
31
|
+
export type ProjectSchedule = 'daily' | 'weekly';
|
|
32
32
|
/**
|
|
33
33
|
* A saved project as the API returns it.
|
|
34
34
|
*
|
|
@@ -81,6 +81,16 @@ export interface ProjectConfig {
|
|
|
81
81
|
scheduleKeyId?: string;
|
|
82
82
|
/** When the scheduler last started a run. The next run is due one interval after this. */
|
|
83
83
|
lastScheduledRunAt?: string;
|
|
84
|
+
/** The standing intent scheduled runs are judged against. Absent means scheduled runs are not judged. */
|
|
85
|
+
intent?: string;
|
|
86
|
+
/**
|
|
87
|
+
* Which environment `testOrigin` is: "production", "staging", "preview", "development",
|
|
88
|
+
* or the team's own word. A label only, shown in emails, the dashboard and MCP; never
|
|
89
|
+
* resets baselines. Absent when unset.
|
|
90
|
+
*/
|
|
91
|
+
environment?: string;
|
|
92
|
+
/** Live-vs-live projects only: which environment `baseOrigin` is. */
|
|
93
|
+
baseEnvironment?: string;
|
|
84
94
|
createdAt: string;
|
|
85
95
|
updatedAt: string;
|
|
86
96
|
lastRunAt?: string;
|
|
@@ -136,15 +146,37 @@ export interface ProjectConfigUpdate {
|
|
|
136
146
|
* into the hour from the following day. Hours only, because the scheduler sweeps once
|
|
137
147
|
* an hour; UTC only, because RegressionBot has no timezone to convert from.
|
|
138
148
|
*
|
|
139
|
-
* Rejected
|
|
140
|
-
*
|
|
141
|
-
* same update. Clearing the schedule with
|
|
149
|
+
* Rejected if the project has no schedule to apply it to — either one already set, or
|
|
150
|
+
* one sent in the same update. Clearing the schedule with
|
|
142
151
|
* `schedule: null` clears this too, so re-enabling later cannot resurrect an hour
|
|
143
152
|
* nobody asked for on that run. Pass null to clear it on its own.
|
|
144
153
|
*
|
|
145
154
|
* A missed slot waits for the next one rather than catching up.
|
|
146
155
|
*/
|
|
147
156
|
scheduleHourUtc?: number | null;
|
|
157
|
+
/**
|
|
158
|
+
* The API key the scheduled runs are attributed to. Ignored when the request itself
|
|
159
|
+
* carries an API key — that key is recorded instead. Required, and checked against the
|
|
160
|
+
* organisation's keys, when a schedule is set by a caller holding no key (the dashboard).
|
|
161
|
+
*/
|
|
162
|
+
scheduleKeyId?: string;
|
|
163
|
+
/**
|
|
164
|
+
* A standing intent for scheduled runs, which have no PR to draw one from. Say what the
|
|
165
|
+
* pages are expected to do between checks and what must not change; every scheduled run
|
|
166
|
+
* is judged against it. Without one a scheduled run reports
|
|
167
|
+
* `intentAssessment.decision: 'not_judged'`. Runs started by a caller are never judged
|
|
168
|
+
* against it. Trimmed; pass null or an empty string to remove. Does not invalidate
|
|
169
|
+
* baselines. Max 1000 characters.
|
|
170
|
+
*/
|
|
171
|
+
intent?: string | null;
|
|
172
|
+
/**
|
|
173
|
+
* Which environment `testOrigin` is: production, staging, preview, development, or your
|
|
174
|
+
* own word (1-24 letters, digits, spaces, hyphens; stored lower-case). A label only:
|
|
175
|
+
* never resets baselines and never changes a run. Pass null or an empty string to remove.
|
|
176
|
+
*/
|
|
177
|
+
environment?: string | null;
|
|
178
|
+
/** Live-vs-live projects only, which environment `baseOrigin` is. Same rules as `environment`. */
|
|
179
|
+
baseEnvironment?: string | null;
|
|
148
180
|
}
|
|
149
181
|
export interface VRConfig {
|
|
150
182
|
apiKey?: string;
|
|
@@ -224,6 +256,10 @@ export interface ResultVerdict {
|
|
|
224
256
|
* content pushed by a change elsewhere — both are consequences, not edits.
|
|
225
257
|
*/
|
|
226
258
|
export type ChangeType = 'text-edit' | 'insert' | 'delete' | 'style-only' | 'image-change' | 'move' | 'move-with-edit' | 'sticky-reposition' | 'reflow-displacement'
|
|
259
|
+
/** A reader-facing attribute changed (a link target, alt text, an aria-label). See `Change.attributes`. */
|
|
260
|
+
| 'attribute-edit'
|
|
261
|
+
/** A web font failed to load for this capture. */
|
|
262
|
+
| 'font-failed'
|
|
227
263
|
/**
|
|
228
264
|
* The six metadata types: the page's head or its structured data changed while nothing
|
|
229
265
|
* visible moved — a title edited, a canonical rewritten, a JSON-LD block dropped. They
|
|
@@ -248,8 +284,7 @@ export interface Change {
|
|
|
248
284
|
type: ChangeType;
|
|
249
285
|
/**
|
|
250
286
|
* Tag name only, lowercase, e.g. `div` — not a CSS selector and not unique on the page.
|
|
251
|
-
* For a metadata type, the field that changed instead.
|
|
252
|
-
* with, see `PageResult.elementsChanged`.
|
|
287
|
+
* For a metadata type, the field that changed instead.
|
|
253
288
|
*/
|
|
254
289
|
element: string;
|
|
255
290
|
/**
|
|
@@ -262,8 +297,29 @@ export interface Change {
|
|
|
262
297
|
before?: string;
|
|
263
298
|
/** The text after the edit. Absent on a delete. Clipped the same way as `before`. */
|
|
264
299
|
after?: string;
|
|
300
|
+
/**
|
|
301
|
+
* On a change with no words of its own (`style-only`, `attribute-edit`, `image-change`):
|
|
302
|
+
* what a reader calls the element, its text or an image's alt text, up to 40 characters
|
|
303
|
+
* ("Apply now"). Absent on a large element such as a section, whose text would not name
|
|
304
|
+
* it, and when the page's changes left no room in the field's size budget: a label never
|
|
305
|
+
* displaces a change.
|
|
306
|
+
*/
|
|
307
|
+
label?: string;
|
|
265
308
|
/** Changed computed styles, keyed by CSS property name. */
|
|
266
309
|
style?: Record<string, StyleDelta>;
|
|
310
|
+
/**
|
|
311
|
+
* On an `attribute-edit`: each changed reader-facing attribute (href, alt, title,
|
|
312
|
+
* aria-label, placeholder, role, type, value) as name to [before, after]. A same-origin
|
|
313
|
+
* link is compared as its path, so a preview's own host is not a change.
|
|
314
|
+
*/
|
|
315
|
+
attributes?: Record<string, StyleDelta>;
|
|
316
|
+
/**
|
|
317
|
+
* Present as `"date"` when every changed word belongs to a date or time ("Sep 17" to
|
|
318
|
+
* "Sep 23", "3 hours ago"). The edit is reported but does not count toward the page
|
|
319
|
+
* having changed, so a page can list one and still read unchanged. Numbers, prices and
|
|
320
|
+
* percentages are never labelled this way.
|
|
321
|
+
*/
|
|
322
|
+
valueOnly?: 'date';
|
|
267
323
|
/**
|
|
268
324
|
* Where it sits in the current capture.
|
|
269
325
|
*
|
|
@@ -298,6 +354,7 @@ export interface RegressionbotSummaryItem {
|
|
|
298
354
|
label: string;
|
|
299
355
|
/** Single-sentence description of what changed in that region. */
|
|
300
356
|
text: string;
|
|
357
|
+
/** @deprecated Never set on current results. To be removed after 2026-10-28. */
|
|
301
358
|
verdict?: RegionVerdict;
|
|
302
359
|
}
|
|
303
360
|
/**
|
|
@@ -335,12 +392,23 @@ export interface PageResult {
|
|
|
335
392
|
* page whose baseline predates metadata capture, which is silence rather than "no".
|
|
336
393
|
*/
|
|
337
394
|
metadataChanged?: true;
|
|
395
|
+
/**
|
|
396
|
+
* Which group this page's change belongs to — `structural`, `edited`, `metadata`,
|
|
397
|
+
* `unmeasured`, `date` or `cosmetic` — so a caller can filter a run to one kind of
|
|
398
|
+
* change without re-deriving the rule. The same value the digest email groups on.
|
|
399
|
+
* Present only on a page that counts as a change. `unmeasured` and `date` occur only
|
|
400
|
+
* on results from before 2026-09-28.
|
|
401
|
+
*/
|
|
402
|
+
changeGroup?: 'structural' | 'edited' | 'metadata' | 'unmeasured' | 'date' | 'cosmetic';
|
|
338
403
|
/**
|
|
339
404
|
* Percentage of pixels that differed from the baseline. Not a change test on its own:
|
|
340
405
|
* 0 does not mean identical — see `changed`.
|
|
341
406
|
*/
|
|
342
407
|
diffPercentage: number;
|
|
343
|
-
/**
|
|
408
|
+
/**
|
|
409
|
+
* @deprecated Deprecated 2026-09-29. 100 minus `diffPercentage`; it no longer measures
|
|
410
|
+
* how alike two pictures look. Kept for existing callers. Read `changed` and `changes`.
|
|
411
|
+
*/
|
|
344
412
|
visualMatchScore: number;
|
|
345
413
|
/** True when this capture became a baseline because none existed to compare against. */
|
|
346
414
|
isNewBaseline: boolean;
|
|
@@ -374,12 +442,13 @@ export interface PageResult {
|
|
|
374
442
|
* not necessarily all of them.
|
|
375
443
|
*/
|
|
376
444
|
changes?: Change[];
|
|
377
|
-
/**
|
|
445
|
+
/** @deprecated Dropped 2026-09-28; already never set before that. Use `changes` instead. */
|
|
378
446
|
elementsChanged?: string[];
|
|
379
447
|
/**
|
|
380
|
-
*
|
|
381
|
-
*
|
|
382
|
-
*
|
|
448
|
+
* @deprecated Only on results from before 2026-09-28, when the pixel engine compared a
|
|
449
|
+
* page without its DOM. To be removed after 2026-10-28, when the last of those expire.
|
|
450
|
+
* The result is a plain pixel diff with no semantic change types, so treat its
|
|
451
|
+
* classifications as absent rather than as "nothing structural changed".
|
|
383
452
|
*/
|
|
384
453
|
domAssistSkipReason?: string;
|
|
385
454
|
/** Pre-signed URL for the stored baseline screenshot. */
|
|
@@ -405,6 +474,10 @@ export interface JobProgress {
|
|
|
405
474
|
export type SummaryStatus = 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'FAILED';
|
|
406
475
|
export interface JobStatus {
|
|
407
476
|
jobId: string;
|
|
477
|
+
/** The project's environment when the run started. A label only. Absent when the project has none. */
|
|
478
|
+
environment?: string;
|
|
479
|
+
/** Live-vs-live runs only: the environment of the base origin. Absent otherwise. */
|
|
480
|
+
baseEnvironment?: string;
|
|
408
481
|
/**
|
|
409
482
|
* Terminal states are COMPLETED, APPROVED, RESOLVED and FAILED. APPROVED means every page
|
|
410
483
|
* carrying a diff was approved; RESOLVED means they were all decided and at least one was
|
|
@@ -418,24 +491,41 @@ export interface JobStatus {
|
|
|
418
491
|
summaryStatus: SummaryStatus;
|
|
419
492
|
error: string | null;
|
|
420
493
|
progress: JobProgress;
|
|
494
|
+
/** The project whose configuration this run used, when it had one. */
|
|
495
|
+
project?: string;
|
|
496
|
+
/** The origin that was captured as "current". */
|
|
497
|
+
origin?: string;
|
|
498
|
+
/** The second origin, captured as the comparison. Present only in live-vs-live mode. */
|
|
499
|
+
baseOrigin?: string;
|
|
421
500
|
executionTime: number;
|
|
422
501
|
createdAt: string;
|
|
423
502
|
/** Whether the run carried an intent. Without one no page gets a verdict. */
|
|
424
503
|
intentProvided?: boolean;
|
|
425
504
|
/** Whether this run compared two live origins or one origin against a stored baseline. */
|
|
426
505
|
comparisonMode?: 'live-vs-live' | 'managed';
|
|
506
|
+
/** Page-checks the run made, and how many finished. Mirrors `progress`. */
|
|
507
|
+
totalUrls?: number;
|
|
508
|
+
completedCount?: number;
|
|
427
509
|
/** Partial or complete results. Fills in as workers finish. */
|
|
428
510
|
results: PageResult[];
|
|
429
511
|
}
|
|
430
512
|
export interface JobSummary {
|
|
431
513
|
jobId: string;
|
|
514
|
+
/** The project's environment when the run started. A label only. Absent when the project has none. */
|
|
515
|
+
environment?: string;
|
|
516
|
+
/** Live-vs-live runs only: the environment of the base origin. Absent otherwise. */
|
|
517
|
+
baseEnvironment?: string;
|
|
432
518
|
/** getSummary() throws until the job reaches one of these. */
|
|
433
519
|
status: 'COMPLETED' | 'APPROVED' | 'RESOLVED' | 'FAILED';
|
|
434
520
|
summaryStatus: SummaryStatus;
|
|
435
521
|
error: string | null;
|
|
436
522
|
totalUrls: number;
|
|
437
523
|
completedCount: number;
|
|
438
|
-
/**
|
|
524
|
+
/**
|
|
525
|
+
* @deprecated Deprecated 2026-09-29. 100 minus the average `diffPercentage` over the
|
|
526
|
+
* job's pages, an errored page counting as 100: not a similarity score. Kept for
|
|
527
|
+
* existing callers. Read `regressionCount` and each regression's `changes`.
|
|
528
|
+
*/
|
|
439
529
|
overallScore: number;
|
|
440
530
|
executionTime: number;
|
|
441
531
|
regressionCount: number;
|
|
@@ -457,6 +547,13 @@ export interface JobSummary {
|
|
|
457
547
|
variantName: string;
|
|
458
548
|
errorMessage: string;
|
|
459
549
|
}>;
|
|
550
|
+
/**
|
|
551
|
+
* An AI summary of the whole run, two or three sentences, written once when the run
|
|
552
|
+
* finished from the exact edits in `regressions` (the same text opens the scheduled-run
|
|
553
|
+
* email). Null when nothing changed or none could be written. The changes are the
|
|
554
|
+
* record; this is a reading of them.
|
|
555
|
+
*/
|
|
556
|
+
runSummary: string | null;
|
|
460
557
|
/** Whole-job roll-up of how the changes line up with the run's stated intent. */
|
|
461
558
|
intentAssessment: IntentAssessment;
|
|
462
559
|
/** The intent this run was given, echoed back. */
|
package/package.json
CHANGED