@regressionbot/sdk 2.0.0 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -57,9 +57,9 @@ The rest surface as type errors:
57
57
 
58
58
  - **`PageResult.diffCount` is gone.** No endpoint ever returned it — both
59
59
  `getJobStatus` and `getJobSummary` strip it. `PageResult` gains the fields they
60
- do return: `status`, `isNewBaseline`, `errorMessage`, `elementsChanged`,
61
- `domAssistSkipReason`, `maskUrl` and `verdict`. Image URLs are typed nullable,
62
- which they always were.
60
+ do return: `status`, `changed`, `isNewBaseline`, `errorMessage`,
61
+ `elementsChanged`, `domAssistSkipReason` and `verdict`. Image URLs are typed
62
+ nullable, which they always were.
63
63
  - **`updateProject()` takes `ProjectConfigUpdate`, not `Partial<ProjectConfig>`.**
64
64
  The read and write shapes genuinely differ: a gate credential is written as an
65
65
  `EnvGate` and read back only as `{ configured: true }`.
@@ -74,6 +74,46 @@ the builder, intent-aware verdicts, environment gates, baseline policies, and
74
74
  scheduling. `scheduleHourUtc` needs API 2.7.0 or later — on an older API the
75
75
  field is accepted and ignored, and the schedule stays anchored to its first run.
76
76
 
77
+ ### 2.0.1 — never published separately; included in 2.1.0
78
+
79
+ Upgrading from 2.0.0 goes straight to 2.1.0, so read this section as well as the
80
+ next one.
81
+
82
+ - **The builder no longer invents defaults.** It used to send `concurrency: 10`
83
+ on every run, plus an empty `devices` list. The API compares every parameter it
84
+ receives against the saved project config and rejects the run on a mismatch, so
85
+ a project storing any other concurrency — including none, which is every project
86
+ created outside the SDK — failed with *"Params differ from stored config"* and
87
+ there was no way to unset the field. Call `.concurrency(n)` only when you mean
88
+ it; unset now means the API's default of 4, not 10.
89
+ - **CLI: `--concurrency` now rejects a missing or non-numeric value** instead of
90
+ coercing it. `--concurrency` with no value used to run at 1, and a
91
+ non-numeric one used to run at 10; both now fail with a message. Passing a
92
+ valid `1`–`20` is unchanged. Misuse exits **2**, so CI can tell a bad flag
93
+ from a run that worked and found regressions — both of which used to exit 1.
94
+ - **`PageResult.maskUrl` is gone.** It was never on the public contract: both
95
+ endpoints emit it to internal callers only, so code reading it was already
96
+ getting `undefined` and nothing changes at runtime. It was listed here in
97
+ 2.0.0 by mistake. There is no public replacement — the mask is a transparent
98
+ overlay built for the dashboard, and `diffUrl` is the annotated diff image you
99
+ want instead.
100
+ - **`PageResult.changed` added**, with `contentChanged` alongside it. This is the
101
+ rule the API uses to split `regressions` from `matches`. Read it instead of
102
+ testing `diffPercentage === 0` — a text edit that moves no pixels is a
103
+ regression at 0.00%.
104
+
105
+ ### 2.1.0
106
+
107
+ - **`--fail-on unintended`** on the CLI: exit 1 only on changes judged a bug or
108
+ needing review, so expected changes keep CI green. Comes with
109
+ `--change-description`, `--expected-changes`, `--pr-title` and `--commit` to
110
+ describe the run. Default behaviour is unchanged.
111
+ - **`--flag=value` is now accepted** for every flag. Required when the value starts
112
+ with `--`, such as a CSS custom property: `--custom-css="--brand: #fff"` used to
113
+ drop its value silently.
114
+ - The run command now reports capture errors even when there are also regressions.
115
+ They used to be hidden behind the regression branch.
116
+
77
117
  ## Usage
78
118
 
79
119
  ### Basic Example
@@ -124,8 +164,9 @@ const job = await rb
124
164
  // Discovery: Auto-discover up to 20 blog posts
125
165
  .scan('/blog/**', { limit: 20 })
126
166
 
127
- // Concurrency: Max parallel browser instances
128
- .concurrency(10)
167
+ // Concurrency: pages captured in parallel, 1-20. Omit it to take the default of 4 —
168
+ // the load lands on the site being captured, not on RegressionBot.
169
+ .concurrency(8)
129
170
 
130
171
  // Masking: Automatic and manual masking
131
172
  .mask(['.ads', '#modal']) // Manual selectors
@@ -170,6 +211,45 @@ const aiResult = await job.generateAiSummary();
170
211
  console.log(`Generated summaries for ${aiResult.summaries.length} regressions.`);
171
212
  ```
172
213
 
214
+ ### Reading exactly what changed
215
+
216
+ `changes` is computed by diffing the two documents, so unlike the prose summary it is
217
+ measured rather than described — you can quote it directly.
218
+
219
+ ```typescript
220
+ for (const r of summary.regressions) {
221
+ for (const c of r.changes ?? []) {
222
+ switch (c.type) {
223
+ case 'text-edit':
224
+ console.log(`${c.element}: "${c.before}" → "${c.after}"`);
225
+ break;
226
+ case 'style-only':
227
+ for (const [prop, [from, to]] of Object.entries(c.style ?? {})) {
228
+ console.log(`${c.element}: ${prop} ${from} → ${to}`);
229
+ }
230
+ break;
231
+ default:
232
+ console.log(`${c.type} on ${c.element}`);
233
+ }
234
+ }
235
+ }
236
+ ```
237
+
238
+ Two things to know before you rely on it:
239
+
240
+ - **It can be truncated.** At most 40 changes per page, and fewer if they would exceed the
241
+ API's 8 KB budget for the field. Treat it as the most significant changes, not a
242
+ guaranteed-complete list.
243
+ - **`changes` absent is not "nothing changed".** It means the document comparison could not
244
+ run on that page — check `domAssistSkipReason`.
245
+
246
+ `box` gives you the location in the capture, in CSS pixels from the top-left of the
247
+ full-page image, so it indexes straight into `currentUrl` if you want to crop:
248
+
249
+ ```typescript
250
+ const aboveTheFold = r.changes?.filter(c => c.box && c.box.y < 800) ?? [];
251
+ ```
252
+
173
253
  ### Intent-Aware Verdicts
174
254
 
175
255
  Tell RegressionBot what a run is meant to change, and every regression comes back judged
@@ -387,7 +467,33 @@ screenshot:
387
467
  npx @regressionbot/sdk https://example.com --project my-site --custom-css "#chat-widget { display: none !important; }"
388
468
  ```
389
469
 
390
- #### 3. Job Summary
470
+ Any flag also accepts `--flag=value`. You need that form when the value itself starts with
471
+ `--`, such as a CSS custom property — otherwise it looks like the next flag and is dropped:
472
+ ```bash
473
+ npx @regressionbot/sdk https://example.com --project my-site --custom-css="--brand-bg: #fff"
474
+ ```
475
+
476
+ #### 3. Failing CI only on unintended changes
477
+
478
+ By default any regression exits 1. Describe what the run is meant to change and
479
+ `--fail-on unintended` will keep the build green for changes that match, failing only on
480
+ what looks like a bug:
481
+
482
+ ```bash
483
+ npx @regressionbot/sdk https://preview.example.com \
484
+ --project my-site \
485
+ --change-description "Restyle the pricing table" \
486
+ --expected-changes "Pricing table is now dark,Buttons are rounded" \
487
+ --commit "$GITHUB_SHA" \
488
+ --fail-on unintended
489
+ ```
490
+
491
+ A change with no verdict always fails the build — if the run carried no intent, or the
492
+ summary pass hasn't finished, nothing judged it, and treating that as expected would turn a
493
+ missing verdict into a silent pass. So `--fail-on unintended` without
494
+ `--change-description` behaves like `--fail-on any`, and warns that it will.
495
+
496
+ #### 4. Job Summary
391
497
  Get detailed results and diff URLs for a completed job.
392
498
  ```bash
393
499
  npx @regressionbot/sdk summary <jobId>
@@ -403,7 +509,7 @@ Use the `--download-full` flag to save baseline, current, and diff images:
403
509
  npx @regressionbot/sdk summary <jobId> --download-full
404
510
  ```
405
511
 
406
- #### 4. Approve Changes
512
+ #### 5. Approve Changes
407
513
  Promote the current screenshots of a job to be the new baselines.
408
514
  ```bash
409
515
  npx @regressionbot/sdk approve <jobId>
package/dist/cli.d.ts CHANGED
@@ -1,2 +1,28 @@
1
1
  #!/usr/bin/env node
2
- export {};
2
+ import { PageResult, RunContext } from './types';
3
+ declare function printRegression(r: PageResult): void;
4
+ type FailOn = 'any' | 'unintended';
5
+ declare function parseFailOn(raw: unknown): FailOn;
6
+ /**
7
+ * Whether one regression should fail the build under `--fail-on unintended`.
8
+ *
9
+ * A regression with no verdict always blocks. Nothing judged it — the run carried no
10
+ * intent, or the summary pass had not finished — and treating unjudged as wanted would
11
+ * turn a missing verdict into a silent pass, which is the one outcome a regression
12
+ * detector must never produce.
13
+ */
14
+ declare function isBlocking(r: PageResult): boolean;
15
+ /**
16
+ * Splits regressions into the ones that fail the build and a count of the ones excused.
17
+ *
18
+ * Pure, and exported, because this is the decision that sets the exit code — the whole
19
+ * point of --fail-on. Testing isBlocking alone proves nothing about whether the caller
20
+ * applies it correctly for each mode.
21
+ */
22
+ declare function selectBlocking(regressions: PageResult[], failOn: FailOn): {
23
+ blocking: PageResult[];
24
+ excused: number;
25
+ };
26
+ declare function buildRunContext(options: any): RunContext | undefined;
27
+ declare function parseArgs(args: string[]): any;
28
+ export { parseArgs, parseFailOn, isBlocking, selectBlocking, buildRunContext, printRegression };
package/dist/cli.js CHANGED
@@ -34,6 +34,12 @@ var __importStar = (this && this.__importStar) || (function () {
34
34
  };
35
35
  })();
36
36
  Object.defineProperty(exports, "__esModule", { value: true });
37
+ exports.parseArgs = parseArgs;
38
+ exports.parseFailOn = parseFailOn;
39
+ exports.isBlocking = isBlocking;
40
+ exports.selectBlocking = selectBlocking;
41
+ exports.buildRunContext = buildRunContext;
42
+ exports.printRegression = printRegression;
37
43
  const index_1 = require("./index");
38
44
  const security_1 = require("./security");
39
45
  const path = __importStar(require("path"));
@@ -42,11 +48,12 @@ function formatSummary(items) {
42
48
  return '';
43
49
  return items
44
50
  .map(item => (item.label ? `[${item.label}] ${item.text}` : item.text))
45
- .join('\n ');
51
+ .join('\n ');
46
52
  }
47
53
  function printRegression(r) {
48
54
  console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
49
- console.log(` Diff: ${r.diffUrl}`);
55
+ if (r.diffUrl)
56
+ console.log(` Diff: ${r.diffUrl}`);
50
57
  if (r.verdict) {
51
58
  console.log(` Verdict: ${r.verdict.decision} (confidence ${r.verdict.minConfidence.toFixed(2)}-${r.verdict.avgConfidence.toFixed(2)})`);
52
59
  }
@@ -54,31 +61,103 @@ function printRegression(r) {
54
61
  if (summary)
55
62
  console.log(` Summary: ${summary}`);
56
63
  }
64
+ /**
65
+ * A mistake in how the command was invoked, rather than a run that worked and found
66
+ * something. Exits 2 so CI can tell the two apart — a pipeline that treats "regressions
67
+ * found" as a real result but "bad flag" as a broken job could not distinguish them when
68
+ * both exited 1.
69
+ */
70
+ class UsageError extends Error {
71
+ }
72
+ function printIntent(summary) {
73
+ const ia = summary.intentAssessment;
74
+ if (!ia || !ia.intentProvided)
75
+ return;
76
+ console.log(`\n🧭 Intent: ${ia.summary}`);
77
+ console.log(` bugs: ${ia.bugCount}, intentional: ${ia.intentionalCount}, noise: ${ia.noiseCount}, needs review: ${ia.needsReviewCount}`);
78
+ }
79
+ function parseFailOn(raw) {
80
+ if (raw === undefined)
81
+ return 'any';
82
+ if (raw === 'any' || raw === 'unintended')
83
+ return raw;
84
+ throw new UsageError("--fail-on takes 'any' or 'unintended'.");
85
+ }
86
+ /**
87
+ * Whether one regression should fail the build under `--fail-on unintended`.
88
+ *
89
+ * A regression with no verdict always blocks. Nothing judged it — the run carried no
90
+ * intent, or the summary pass had not finished — and treating unjudged as wanted would
91
+ * turn a missing verdict into a silent pass, which is the one outcome a regression
92
+ * detector must never produce.
93
+ */
94
+ function isBlocking(r) {
95
+ if (!r.verdict)
96
+ return true;
97
+ return r.verdict.decision === 'bug' || r.verdict.decision === 'needs_review';
98
+ }
99
+ /**
100
+ * Splits regressions into the ones that fail the build and a count of the ones excused.
101
+ *
102
+ * Pure, and exported, because this is the decision that sets the exit code — the whole
103
+ * point of --fail-on. Testing isBlocking alone proves nothing about whether the caller
104
+ * applies it correctly for each mode.
105
+ */
106
+ function selectBlocking(regressions, failOn) {
107
+ const blocking = failOn === 'any' ? regressions : regressions.filter(isBlocking);
108
+ return { blocking, excused: regressions.length - blocking.length };
109
+ }
110
+ function buildRunContext(options) {
111
+ const context = {};
112
+ if (typeof options['change-description'] === 'string')
113
+ context.changeDescription = options['change-description'];
114
+ if (typeof options['pr-title'] === 'string')
115
+ context.prTitle = options['pr-title'];
116
+ if (typeof options.commit === 'string')
117
+ context.gitCommitSha = options.commit;
118
+ if (typeof options['expected-changes'] === 'string') {
119
+ const expected = options['expected-changes'].split(',').map((s) => s.trim()).filter(Boolean);
120
+ if (expected.length > 0)
121
+ context.expectedChanges = expected;
122
+ }
123
+ return Object.keys(context).length > 0 ? context : undefined;
124
+ }
125
+ // 🛡️ SECURITY: never let an argument name reach Object.prototype.
126
+ const BLOCKED_KEYS = new Set(['__proto__', 'constructor', 'prototype']);
57
127
  function parseArgs(args) {
58
128
  const options = Object.create(null);
59
129
  options._ = [];
60
130
  for (let i = 0; i < args.length; i++) {
61
131
  const arg = args[i];
62
- if (arg.startsWith('--')) {
63
- const key = arg.slice(2);
64
- // Prevent prototype pollution
65
- if (key === '__proto__' || key === 'constructor' || key === 'prototype') {
66
- const value = args[i + 1];
67
- if (value && !value.startsWith('--'))
68
- i++;
69
- continue;
70
- }
132
+ if (!arg.startsWith('--')) {
133
+ options._.push(arg);
134
+ continue;
135
+ }
136
+ const body = arg.slice(2);
137
+ const eq = body.indexOf('=');
138
+ // --key=value carries its own value, so the value survives even when it starts
139
+ // with '--'. That is the only way to pass CSS custom properties:
140
+ // `--custom-css="--brand: red"` loses its value in the space-separated form,
141
+ // because a leading '--' is indistinguishable from the next flag.
142
+ if (eq !== -1) {
143
+ const key = body.slice(0, eq);
144
+ if (!BLOCKED_KEYS.has(key))
145
+ options[key] = body.slice(eq + 1);
146
+ continue;
147
+ }
148
+ if (BLOCKED_KEYS.has(body)) {
71
149
  const value = args[i + 1];
72
- if (value && !value.startsWith('--')) {
73
- options[key] = value;
150
+ if (value && !value.startsWith('--'))
74
151
  i++;
75
- }
76
- else {
77
- options[key] = true;
78
- }
152
+ continue;
153
+ }
154
+ const value = args[i + 1];
155
+ if (value && !value.startsWith('--')) {
156
+ options[body] = value;
157
+ i++;
79
158
  }
80
159
  else {
81
- options._.push(arg);
160
+ options[body] = true;
82
161
  }
83
162
  }
84
163
  return options;
@@ -121,7 +200,7 @@ async function main() {
121
200
  catch (error) {
122
201
  console.error(`
123
202
  Error: ${error.message}`);
124
- process.exit(1);
203
+ process.exit(error instanceof UsageError ? 2 : 1);
125
204
  }
126
205
  }
127
206
  function showHelp() {
@@ -143,24 +222,56 @@ Options for <url>:
143
222
  --on <devices> Comma-separated device names (e.g. "Desktop Chrome,iPhone 12").
144
223
  --scan <pattern> Glob pattern to scan in sitemap (e.g. "/blog/**").
145
224
  --exclude <patterns> Comma-separated glob patterns to exclude.
146
- --concurrency <n> Max concurrent workers (default 10).
225
+ --concurrency <n> Pages captured in parallel, 1-20 (default 4).
147
226
  --auto-approve Automatically approve results as new baselines.
148
227
  --mask <selectors> Comma-separated CSS selectors to hide (e.g. ".ad,#popup").
149
228
  --custom-css <css> CSS injected before each screenshot (max 4096 chars).
229
+ Use --custom-css="..." if the CSS starts with '--'.
150
230
  --skip-summaries Skip waiting for parallel AI summaries in the CLI.
151
231
 
232
+ Describing the change (lets each regression be judged against your intent):
233
+ --change-description <text> What this run is meant to change.
234
+ --expected-changes <list> Comma-separated list of expected changes.
235
+ --pr-title <text> PR title.
236
+ --commit <sha> Git commit SHA.
237
+
238
+ --fail-on <mode> What counts as a failure. 'any' (default) fails on any
239
+ regression. 'unintended' fails only on changes judged a
240
+ bug or needing review, so expected changes keep the build
241
+ green. Needs --change-description; anything unjudged
242
+ still fails.
243
+
244
+ Note: any flag can be written --flag=value, which is required when the value
245
+ itself starts with '--'.
246
+
152
247
  Environment Variables:
153
248
  REGRESSIONBOT_API_KEY Override the API Key.
154
249
  REGRESSIONBOT_API_URL Override the API URL.
250
+
251
+ Exit codes:
252
+ 0 Nothing failed (see --fail-on).
253
+ 1 Regressions or capture errors found, or the run failed.
254
+ 2 The command was used incorrectly (bad flag or missing argument).
155
255
  `);
156
256
  }
157
257
  async function startJob(url, options) {
158
258
  console.log(`🚀 Initializing visual test...`);
159
259
  const projectId = options.project;
160
- const devices = options.on ? options.on.split(',').map((s) => s.trim()) : ['Desktop Chrome'];
161
- const builder = sdk.test(url)
162
- .on(devices)
163
- .concurrency(Number(options.concurrency) || 10);
260
+ const failOn = parseFailOn(options['fail-on']);
261
+ const builder = sdk.test(url);
262
+ // Only set what the caller actually asked for. Anything sent here is compared against
263
+ // the saved project config, so a default invented by the CLI fails the run instead of
264
+ // being overridden by it.
265
+ if (typeof options.on === 'string') {
266
+ builder.on(options.on.split(',').map((s) => s.trim()).filter(Boolean));
267
+ }
268
+ if (options.concurrency !== undefined) {
269
+ const n = typeof options.concurrency === 'string' ? Number(options.concurrency) : NaN;
270
+ if (!Number.isInteger(n) || n < 1 || n > 20) {
271
+ throw new UsageError('--concurrency takes a whole number from 1 to 20.');
272
+ }
273
+ builder.concurrency(n);
274
+ }
164
275
  if (projectId) {
165
276
  builder.forProject(projectId);
166
277
  }
@@ -188,10 +299,20 @@ async function startJob(url, options) {
188
299
  if (typeof options['custom-css'] === 'string') {
189
300
  builder.customCss(options['custom-css']);
190
301
  }
302
+ const runContext = buildRunContext(options);
303
+ if (runContext) {
304
+ builder.withContext(runContext);
305
+ }
306
+ if (failOn === 'unintended' && !runContext) {
307
+ console.log('⚠️ --fail-on unintended needs intent to judge against. Pass --change-description (see --help).');
308
+ }
309
+ if (failOn === 'unintended' && options['skip-summaries']) {
310
+ console.log('⚠️ --skip-summaries leaves every change unjudged, so --fail-on unintended cannot excuse any of them.');
311
+ }
191
312
  const job = await builder.run();
192
313
  console.log(`✅ Job started! ID: ${job.jobId}`);
193
314
  console.log(`📊 Project: ${projectId}`);
194
- console.log(`📱 Matrix: ${devices.join(', ')}`);
315
+ console.log(`📱 Matrix: ${typeof options.on === 'string' ? options.on : 'Desktop Chrome (default)'}`);
195
316
  if (options.scan) {
196
317
  console.log(`🔍 Scan: ${options.scan} (Exclude: ${options.exclude || 'none'})`);
197
318
  }
@@ -216,22 +337,36 @@ Waiting for completion...
216
337
  console.log(`- ${nb.url} [${nb.variantName}]`);
217
338
  });
218
339
  }
340
+ let exitCode = 0;
219
341
  if (summary.regressionCount > 0) {
220
342
  console.log('\n❌ Regressions found:');
221
343
  summary.regressions.forEach(printRegression);
222
- console.log(`\nTo approve these changes, run:\n npx regressionbot approve ${job.jobId}`);
223
- process.exit(1);
344
+ const { blocking, excused } = selectBlocking(summary.regressions, failOn);
345
+ if (excused > 0) {
346
+ console.log(`\n✅ ${excused} of ${summary.regressionCount} matched the stated intent and did not fail the build.`);
347
+ }
348
+ if (blocking.length > 0) {
349
+ console.log(`\nTo approve these changes, run:\n npx regressionbot approve ${job.jobId}`);
350
+ exitCode = 1;
351
+ }
224
352
  }
225
- else if (summary.errorCount > 0) {
353
+ printIntent(summary);
354
+ // Reported even when there are regressions. These used to be hidden behind an
355
+ // else-if, so a run with both showed only the regressions and exited 1 for what
356
+ // looked like the wrong reason.
357
+ if (summary.errorCount > 0) {
226
358
  console.log('\n⚠️ Errors encountered:');
227
359
  summary.errors.forEach((e) => {
228
360
  console.log(`- ${e.url}: ${e.errorMessage}`);
229
361
  });
230
- process.exit(1);
362
+ exitCode = 1;
231
363
  }
232
- else {
233
- console.log('\n✨ No regressions found. All good!');
364
+ if (exitCode !== 0) {
365
+ process.exit(exitCode);
234
366
  }
367
+ console.log(summary.regressionCount > 0
368
+ ? '\n✨ Every change was intended. All good!'
369
+ : '\n✨ No regressions found. All good!');
235
370
  }
236
371
  async function checkStatus(jobId) {
237
372
  const job = sdk.job(jobId);
@@ -255,11 +390,7 @@ Errors: ${summary.errorCount}
255
390
  console.log('❌ Regressions found:');
256
391
  summary.regressions.forEach(printRegression);
257
392
  }
258
- if (summary.intentAssessment) {
259
- const ia = summary.intentAssessment;
260
- console.log(`\n🧭 Intent: ${ia.summary}`);
261
- console.log(` bugs: ${ia.bugCount}, intentional: ${ia.intentionalCount}, noise: ${ia.noiseCount}, needs review: ${ia.needsReviewCount}`);
262
- }
393
+ printIntent(summary);
263
394
  const doDownload = options.download || options['download-full'];
264
395
  if (doDownload) {
265
396
  console.log(`\n💾 Downloading results...`);
@@ -281,4 +412,8 @@ async function approveJob(jobId) {
281
412
  const res = await job.approve();
282
413
  console.log(`Success! ${res.message}`);
283
414
  }
284
- main();
415
+ // Only run when invoked as a command, so the pure helpers below can be imported and
416
+ // tested without the CLI executing on require.
417
+ if (require.main === module) {
418
+ main();
419
+ }
package/dist/index.d.ts CHANGED
@@ -1,7 +1,7 @@
1
- import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult } from './types';
1
+ import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult, Change, ChangeType, StyleDelta, ChangeBox } from './types';
2
2
  import { sanitizeFilename, sanitizeUrlToPath } from './security';
3
3
  export { sanitizeFilename, sanitizeUrlToPath, Viewports };
4
- export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult };
4
+ export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult, Change, ChangeType, StyleDelta, ChangeBox };
5
5
  export declare class RegressionBot {
6
6
  private apiKey;
7
7
  private apiUrl;
@@ -78,6 +78,11 @@ export declare class JobBuilder {
78
78
  limit?: number;
79
79
  exclude?: string[];
80
80
  }): this;
81
+ /**
82
+ * How many pages are captured in parallel, 1–20. Leave it unset to take the API's
83
+ * default of 4 — the ceiling is the site being captured, not RegressionBot, so
84
+ * raising it is a load decision about someone else's environment.
85
+ */
81
86
  concurrency(n: number): this;
82
87
  autoApprove(val?: boolean): this;
83
88
  mask(selectors: string[]): this;
package/dist/index.js CHANGED
@@ -94,8 +94,7 @@ class JobBuilder {
94
94
  testOrigin: testOrigin.replace(/\/$/, ''),
95
95
  variants: [],
96
96
  checks: [],
97
- scans: [],
98
- concurrency: 10
97
+ scans: []
99
98
  };
100
99
  }
101
100
  against(origin) {
@@ -133,6 +132,11 @@ class JobBuilder {
133
132
  this.manifest.scans.push({ pattern, options });
134
133
  return this;
135
134
  }
135
+ /**
136
+ * How many pages are captured in parallel, 1–20. Leave it unset to take the API's
137
+ * default of 4 — the ceiling is the site being captured, not RegressionBot, so
138
+ * raising it is a load decision about someone else's environment.
139
+ */
136
140
  concurrency(n) {
137
141
  this.manifest.concurrency = n;
138
142
  return this;
@@ -173,14 +177,18 @@ class JobBuilder {
173
177
  if (this.manifest.checks.length === 0 && this.manifest.scans.length === 0) {
174
178
  this.manifest.checks.push({ path: '/', label: 'Home' });
175
179
  }
180
+ // Every field present here is compared against the saved project config, and a
181
+ // value that differs fails the run rather than overriding it. So a field the
182
+ // caller never set must stay absent: an empty `devices` or a `concurrency` this
183
+ // builder picked on its own would be read as an opinion and diffed.
176
184
  const payload = {
177
185
  project: this.manifest.projectId,
178
186
  testOrigin: this.manifest.testOrigin,
179
187
  sitemapUrl: this.manifest.sitemapUrl,
180
188
  baseOrigin: this.manifest.baseOrigin,
181
- devices: this.manifest.variants,
189
+ devices: this.manifest.variants.length > 0 ? this.manifest.variants : undefined,
182
190
  paths: this.manifest.checks,
183
- scans: this.manifest.scans,
191
+ scans: this.manifest.scans.length > 0 ? this.manifest.scans : undefined,
184
192
  concurrency: this.manifest.concurrency,
185
193
  autoApprove: this.manifest.autoApprove,
186
194
  masks: this.manifest.masks,
package/dist/types.d.ts CHANGED
@@ -55,8 +55,9 @@ export interface ProjectConfig {
55
55
  baseAuth?: {
56
56
  configured: true;
57
57
  };
58
+ /** Pages captured in parallel, 1–20. Absent means the API's default of 4. */
58
59
  concurrency?: number;
59
- /** Extra instructions handed to the model that writes change summaries. */
60
+ /** Extra instructions handed to the model that writes change summaries. Max 1000 characters. */
60
61
  aiPromptInstructions?: string;
61
62
  /** Diff percentage below which a regression is skipped by the AI pass. Defaults to 0.01. */
62
63
  aiSummaryThreshold?: number;
@@ -110,7 +111,12 @@ export interface ProjectConfigUpdate {
110
111
  testAuth?: EnvGate | null;
111
112
  /** Credentials for the base origin's environment gate. Pass null to clear. */
112
113
  baseAuth?: EnvGate | null;
114
+ /**
115
+ * Pages captured in parallel, 1–20. Rejected outside that range. Leave it unset to
116
+ * take the API's default of 4 — the load lands on the site being captured.
117
+ */
113
118
  concurrency?: number;
119
+ /** Max 1000 characters. */
114
120
  aiPromptInstructions?: string;
115
121
  aiSummaryThreshold?: number;
116
122
  baselinePolicy?: BaselinePolicy;
@@ -181,6 +187,71 @@ export interface ResultVerdict {
181
187
  avgConfidence: number;
182
188
  /** Keyed by region label (A, B, C…). Empty when the verdict came from the text-only pass. */
183
189
  regions: Record<string, RegionVerdict>;
190
+ /**
191
+ * What the judgement was made from.
192
+ *
193
+ * `measured` — the exact edits from the document diff were in front of the model.
194
+ * `described` — only a generated sentence about the change was, because the DOM engine
195
+ * could not run on that page. Absent on the vision path, which reads the images.
196
+ *
197
+ * Worth more than the confidence numbers when deciding how much weight to give a
198
+ * verdict: a `described` one was reached without seeing what actually changed.
199
+ */
200
+ basis?: 'measured' | 'described';
201
+ }
202
+ /**
203
+ * The kinds of change the document diff reports.
204
+ *
205
+ * `move` and `move-with-edit` are distinguished because an element that only shifted is
206
+ * usually a knock-on effect, while one that shifted *and* changed is usually the cause.
207
+ * `sticky-reposition` is a move of a positioned element, and `reflow-displacement` is
208
+ * content pushed by a change elsewhere — both are consequences, not edits.
209
+ */
210
+ export type ChangeType = 'text-edit' | 'insert' | 'delete' | 'style-only' | 'image-change' | 'move' | 'move-with-edit' | 'sticky-reposition' | 'reflow-displacement';
211
+ /** One changed CSS property, as `[from, to]`. */
212
+ export type StyleDelta = [from: string, to: string];
213
+ /** A rectangle in the capture, in CSS pixels. See {@link Change.box}. */
214
+ export interface ChangeBox {
215
+ x: number;
216
+ y: number;
217
+ w: number;
218
+ h: number;
219
+ }
220
+ /**
221
+ * One change on a page, computed by diffing the two documents rather than described by a
222
+ * model. This is the exact answer to "what changed" — read it before the prose summary.
223
+ */
224
+ export interface Change {
225
+ type: ChangeType;
226
+ /**
227
+ * Tag name only, lowercase, e.g. `div` — not a CSS selector and not unique on the page.
228
+ * For something you can query with, see `PageResult.elementsChanged`.
229
+ */
230
+ element: string;
231
+ /** The text before the edit. Absent on an insert. */
232
+ before?: string;
233
+ /** The text after the edit. Absent on a delete. */
234
+ after?: string;
235
+ /** Changed computed styles, keyed by CSS property name. */
236
+ style?: Record<string, StyleDelta>;
237
+ /**
238
+ * Where it sits in the current capture.
239
+ *
240
+ * CSS pixels, which equal image pixels here because captures pin `deviceScaleFactor`
241
+ * to 1, and measured from the top-left of the **full-page** capture rather than the
242
+ * viewport — so it indexes straight into `currentUrl` with no scroll offset to add.
243
+ *
244
+ * Absent on a delete, and absent whenever the element's position cannot be trusted:
245
+ * an out-of-flow element reports a viewport-relative rect that would point at empty
246
+ * space in a full-page image, so the box is omitted rather than sent wrong.
247
+ */
248
+ box?: ChangeBox;
249
+ /**
250
+ * Where it was in the baseline, in the same coordinate space, present only when it
251
+ * differs from `box` in position or size. Subtract for the move vector
252
+ * (`box.x - boxBefore.x`); compare `w`/`h` for a resize.
253
+ */
254
+ boxBefore?: ChangeBox;
184
255
  }
185
256
  export interface RegressionbotSummaryItem {
186
257
  /** Region letter (A, B, C…), or an empty string for a whole-page change. */
@@ -202,7 +273,26 @@ export interface PageResult {
202
273
  status: 'SUCCESS' | 'ERROR';
203
274
  /** Device variant tested (e.g. "Desktop Chrome", "iPhone 12"). */
204
275
  variantName: string;
205
- /** Percentage of pixels that differed from the baseline (0 = identical). */
276
+ /**
277
+ * Whether this page is considered changed. Read this rather than deriving it from
278
+ * `diffPercentage` — it is the same rule the API uses to split `regressions` from
279
+ * `matches`, and a text edit that moved no pixels is changed at 0.00%.
280
+ *
281
+ * Required, not optional: both endpoints compute it on every result rather than
282
+ * copying it from storage, so it cannot be missing from a response. The SDK targets
283
+ * the hosted API at api.regressionbot.com, so there is no older deployment that
284
+ * could omit it.
285
+ */
286
+ changed: boolean;
287
+ /**
288
+ * Set when the document diff found a content change. Present only when true, and the
289
+ * reason a page can be `changed` at a `diffPercentage` of 0.
290
+ */
291
+ contentChanged?: true;
292
+ /**
293
+ * Percentage of pixels that differed from the baseline. Not a change test on its own:
294
+ * 0 does not mean identical — see `changed`.
295
+ */
206
296
  diffPercentage: number;
207
297
  /** Perceptual similarity 0-100 (SSIM). */
208
298
  visualMatchScore: number;
@@ -217,6 +307,19 @@ export interface PageResult {
217
307
  * the run carried a {@link RunContext}, and only once summaryStatus is COMPLETE.
218
308
  */
219
309
  verdict?: ResultVerdict;
310
+ /**
311
+ * The exact edits the engine computed from the two documents — element, text before,
312
+ * text after, changed styles. Absent when the DOM comparison could not run on the page,
313
+ * which is not the same as nothing having changed.
314
+ *
315
+ * Prefer this over `regressionbotSummary`: it is measured rather than described, so it
316
+ * carries no confidence and can be quoted directly.
317
+ *
318
+ * Truncated on a busy page: at most 40 changes, and fewer if they would exceed the
319
+ * API's 8 KB budget for this field. Treat the list as the most significant changes,
320
+ * not necessarily all of them.
321
+ */
322
+ changes?: Change[];
220
323
  /** Selectors of the elements that changed, when the DOM comparison could identify them. */
221
324
  elementsChanged?: string[];
222
325
  /**
@@ -231,11 +334,6 @@ export interface PageResult {
231
334
  currentUrl: string | null;
232
335
  /** Pre-signed URL for the side-by-side annotated diff image. */
233
336
  diffUrl: string | null;
234
- /**
235
- * Pre-signed URL for the diff mask. getStatus() always sends the key, null when
236
- * there is no mask; getSummary() omits it entirely in that case.
237
- */
238
- maskUrl?: string | null;
239
337
  }
240
338
  export interface JobProgress {
241
339
  total: number;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@regressionbot/sdk",
3
- "version": "2.0.0",
3
+ "version": "2.1.0",
4
4
  "description": "The official SDK for regressionbot.com - the simplest way to automate visual regression testing.",
5
5
  "homepage": "https://regressionbot.com",
6
6
  "repository": {