@regressionbot/sdk 1.2.2 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,11 +4,16 @@ The official SDK for [RegressionBot.com](https://regressionbot.com) — the simp
4
4
 
5
5
  RegressionBot is a declarative visual regression testing platform that helps you catch UI changes before they reach production. This SDK provides a fluent, chainable API to define your test scope, run visual tests, and manage baselines programmatically.
6
6
 
7
+ [![RegressionBot Docs](https://img.shields.io/badge/docs-regressionbot.com-4be277?style=for-the-badge&labelColor=0b0f14)](https://regressionbot.com/docs)
8
+
9
+ ---
10
+
11
+
7
12
  ## Why RegressionBot?
8
13
 
9
14
  Unlike traditional visual diffing libraries, RegressionBot is designed for modern, automated development loops and agentic pipelines:
10
15
 
11
- - **Highly Accurate Regressions (Zero Noise)**: Leveraging advanced pixel-matching algorithms and element masking (using CSS selectors or the automatic `data-vr-mask` attribute), RegressionBot eliminates false positives caused by dynamic data, layout shifting, or third-party widgets.
16
+ - **Highly Accurate Regressions (Less Noise)**: Leveraging advanced pixel-matching algorithms and element masking (using CSS selectors or the automatic `data-vr-mask` attribute), RegressionBot minimizes false positives caused by dynamic data, layout shifting, or third-party widgets.
12
17
  - **Plain-English Summaries**: No more manual screenshot comparisons. RegressionBot translates visual diffs into concise, plain-English descriptions of what changed, so you know exactly what was modified at a glance.
13
18
  - **Agentic Workflow Ready**: Built from the ground up to support autonomous coding agents (like Gemini or Claude) and automated developer loops. Through standard API endpoints, CLI commands, and Model Context Protocol (MCP) integrations, agents can trigger tests, read plain-English results, and approve baseline changes programmatically without human intervention.
14
19
 
@@ -19,6 +24,8 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
19
24
  - **Matrix Testing**: Test multiple devices and viewports in a single job.
20
25
  - **Auto-Discovery**: Scan sitemaps with glob patterns and limits.
21
26
  - **RegressionBot Summaries**: Plain-English change descriptions for every regression, generated on-demand via the API.
27
+ - **Intent-Aware Verdicts**: Describe what a run is meant to change, and every regression comes back judged as intentional, a bug, or noise.
28
+ - **Scheduled Checks**: Run a saved project unattended, hourly, daily, or weekly — and at a set UTC hour.
22
29
  - **Project-Based Baselines**: Save and reuse test configurations; share visual history across environments.
23
30
  - **Auto-Approval**: Automatically promote screenshots to baselines on jobs that pass your criteria.
24
31
  - **Zero Infrastructure**: No browser maintenance or server provisioning — RegressionBot handles it all.
@@ -29,6 +36,44 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
29
36
  npm install @regressionbot/sdk
30
37
  ```
31
38
 
39
+ ## Migration from 1.x
40
+
41
+ 2.0.0 re-types the SDK against what the API actually returns, taken from the
42
+ orchestrator's implementation rather than from a spec that had drifted in both
43
+ directions. Most of the breakage is caught by the compiler. One is not:
44
+
45
+ ```typescript
46
+ // 1.x — updateProject() returned the raw { message, project } envelope,
47
+ // so every field read off it was undefined at runtime
48
+ const project = await rb.updateProject('site', { schedule: 'daily' });
49
+ project.project.schedule; // this is where the value actually was
50
+
51
+ // 2.0 — the envelope is unwrapped for you
52
+ const project = await rb.updateProject('site', { schedule: 'daily' });
53
+ project.schedule; // 'daily'
54
+ ```
55
+
56
+ The rest surface as type errors:
57
+
58
+ - **`PageResult.diffCount` is gone.** No endpoint ever returned it — both
59
+ `getJobStatus` and `getJobSummary` strip it. `PageResult` gains the fields they
60
+ do return: `status`, `isNewBaseline`, `errorMessage`, `elementsChanged`,
61
+ `domAssistSkipReason`, `maskUrl` and `verdict`. Image URLs are typed nullable,
62
+ which they always were.
63
+ - **`updateProject()` takes `ProjectConfigUpdate`, not `Partial<ProjectConfig>`.**
64
+ The read and write shapes genuinely differ: a gate credential is written as an
65
+ `EnvGate` and read back only as `{ configured: true }`.
66
+ - **`ProjectConfig.orgId` is gone** — the API stopped returning it.
67
+ - **`summaryStatus` can be `FAILED`**, which the old union omitted. `error`,
68
+ `progress`, `results`, `createdAt` and `intentAssessment` are always returned.
69
+ - **`approve()` gains `conflictedUrls`**, returned when another job updated a
70
+ baseline first.
71
+
72
+ New in 2.0 and requiring a current API: `.customCss()` and `.withContext()` on
73
+ the builder, intent-aware verdicts, environment gates, baseline policies, and
74
+ scheduling. `scheduleHourUtc` needs API 2.7.0 or later — on an older API the
75
+ field is accepted and ignored, and the schedule stays anchored to its first run.
76
+
32
77
  ## Usage
33
78
 
34
79
  ### Basic Example
@@ -86,6 +131,12 @@ const job = await rb
86
131
  .mask(['.ads', '#modal']) // Manual selectors
87
132
  // Tip: Adding 'data-vr-mask' to your HTML elements masks them automatically!
88
133
 
134
+ // Custom CSS: Injected before every screenshot (max 4096 chars)
135
+ .customCss('#chat-widget { display: none !important; }')
136
+
137
+ // Intent: What this run is testing, so changes can be judged against it
138
+ .withContext({ changeDescription: 'Restyle the pricing table' })
139
+
89
140
  // Execute: Compiles manifest and triggers the API
90
141
  .run();
91
142
 
@@ -107,9 +158,9 @@ const summary = await job.getSummary();
107
158
  if (summary.regressions.length > 0) {
108
159
  console.log(`\n${summary.regressions.length} regressions found:`);
109
160
  for (const regression of summary.regressions) {
110
- if (regression.regressionbotSummary) {
111
- console.log(`\nRegressionBot Summary for ${regression.url}:`);
112
- console.log(`> ${regression.regressionbotSummary}`);
161
+ for (const item of regression.regressionbotSummary ?? []) {
162
+ // `label` is the region letter (A, B, C…), or '' for a whole-page change
163
+ console.log(`[${item.label}] ${item.text}`);
113
164
  }
114
165
  }
115
166
  }
@@ -119,6 +170,68 @@ const aiResult = await job.generateAiSummary();
119
170
  console.log(`Generated summaries for ${aiResult.summaries.length} regressions.`);
120
171
  ```
121
172
 
173
+ ### Intent-Aware Verdicts
174
+
175
+ Tell RegressionBot what a run is meant to change, and every regression comes back judged
176
+ against that intent — `intentional`, `bug`, `noise`, or `needs_review`.
177
+
178
+ ```typescript
179
+ const job = await rb
180
+ .test(process.env.VERCEL_PREVIEW_URL)
181
+ .forProject('marketing-site-v2')
182
+ .withContext({
183
+ changeDescription: 'Restyle the pricing table',
184
+ gitCommitSha: process.env.GITHUB_SHA,
185
+ prTitle: process.env.PR_TITLE,
186
+ expectedChanges: ['Pricing table background is now dark'],
187
+ scope: ['components/PricingTable.tsx'],
188
+ })
189
+ .run();
190
+
191
+ await job.waitForCompletion(2000, undefined, { waitForSummaries: true });
192
+ const summary = await job.getSummary();
193
+
194
+ // Whole-job roll-up: does everything that changed line up with the stated intent?
195
+ console.log(summary.intentAssessment?.summary);
196
+ console.log(`Bugs: ${summary.intentAssessment?.bugCount}`);
197
+
198
+ // Per-result verdict, plus the per-region verdicts behind it
199
+ for (const r of summary.regressions) {
200
+ console.log(`${r.url}: ${r.verdict?.decision} (${r.verdict?.avgConfidence})`);
201
+ }
202
+
203
+ // Fail CI only on changes that were not intended
204
+ const bugs = summary.regressions.filter(r => r.verdict?.decision === 'bug');
205
+ if (bugs.length > 0) process.exit(1);
206
+ ```
207
+
208
+ `runProject()` takes the same context, so a run from a saved project can be
209
+ intent-aware too:
210
+
211
+ ```typescript
212
+ const job = await rb.runProject('marketing-site-v2', {
213
+ testOrigin: process.env.VERCEL_PREVIEW_URL,
214
+ runContext: { changeDescription: 'Restyle the pricing table' },
215
+ });
216
+ ```
217
+
218
+ ### Hiding Dynamic Content
219
+
220
+ `.mask()` hides elements by selector. `.customCss()` injects arbitrary CSS before each
221
+ screenshot when masking isn't enough — collapsing an animation, pinning a carousel, or
222
+ neutralising a third-party widget. Max 4096 characters.
223
+
224
+ ```typescript
225
+ const job = await rb
226
+ .test('https://preview.myapp.com')
227
+ .forProject('my-app-web')
228
+ .customCss(`
229
+ #chat-widget { display: none !important; }
230
+ * { animation: none !important; transition: none !important; }
231
+ `)
232
+ .run();
233
+ ```
234
+
122
235
  ### Progress Tracking
123
236
 
124
237
  Pass a callback to `waitForCompletion` to receive status updates while the job runs:
@@ -172,6 +285,69 @@ const summary = await job.getSummary();
172
285
  console.log(`Score: ${summary.overallScore}/100`);
173
286
  ```
174
287
 
288
+ ### Environment Gates
289
+
290
+ If an origin sits behind basic auth, a bypass header, or a session cookie, store the
291
+ credential on the project. It is encrypted at rest and never returned — reads report
292
+ only `{ configured: true }`.
293
+
294
+ ```typescript
295
+ await rb.updateProject('marketing-site-v2', {
296
+ testAuth: { basic: { username: 'preview', password: process.env.PREVIEW_PASSWORD! } },
297
+ // or: { headers: { 'x-vercel-protection-bypass': token } }
298
+ // or: { cookies: [{ name: 'session', value: token }] }
299
+ });
300
+
301
+ const project = await rb.getProject('marketing-site-v2');
302
+ console.log(project.testAuth); // { configured: true }
303
+
304
+ // Clear it
305
+ await rb.updateProject('marketing-site-v2', { testAuth: null });
306
+ ```
307
+
308
+ Setting or clearing a gate credential invalidates the stored baselines, since it changes
309
+ what the capture can reach.
310
+
311
+ ### Scheduled Checks
312
+
313
+ A project can run unattended on a fixed cadence. Scheduling requires
314
+ `baselinePolicy: 'rolling'` on a managed project — on the default `approved` policy an
315
+ unapproved change would be re-reported on every subsequent run, so the API rejects the
316
+ combination. Live-vs-live projects store no baseline and are exempt.
317
+
318
+ ```typescript
319
+ // Turn on a daily unattended check
320
+ await rb.updateProject('marketing-site-v2', {
321
+ baselinePolicy: 'rolling',
322
+ schedule: 'daily', // 'hourly' | 'daily' | 'weekly'
323
+ });
324
+
325
+ // Or pin it to a UTC hour — this one runs at 03:00 UTC every day
326
+ await rb.updateProject('marketing-site-v2', {
327
+ baselinePolicy: 'rolling',
328
+ schedule: 'daily',
329
+ scheduleHourUtc: 3, // 0–23, UTC
330
+ });
331
+
332
+ // Turn it back off (this clears scheduleHourUtc too)
333
+ await rb.updateProject('marketing-site-v2', { schedule: null });
334
+
335
+ // Read back when the scheduler last fired — the next run is due one interval after this
336
+ const project = await rb.getProject('marketing-site-v2');
337
+ console.log(project.schedule, project.scheduleHourUtc, project.lastScheduledRunAt);
338
+ ```
339
+
340
+ Without `scheduleHourUtc` the first run starts at the next hourly sweep and the cadence
341
+ anchors to it. With it, the first run waits for that hour as well. Hours only, since the
342
+ scheduler sweeps once an hour, and UTC only. It is rejected on an `hourly` schedule, and
343
+ rejected on a project with no schedule to apply it to. Clearing the schedule clears the
344
+ hour with it, and a missed slot waits for the next one rather than catching up.
345
+
346
+ Setting a schedule, its hour, or a baseline policy does **not** invalidate
347
+ baselines — only changes that affect what a capture looks like (`testOrigin`,
348
+ `baseOrigin`, `sitemapUrl`, `paths`, `scans`, `devices`, `masks`, `customCss`) do that.
349
+ Billing is per comparison, so cost scales with frequency.
350
+
175
351
  ### Reconnecting to an Existing Job
176
352
 
177
353
  If you have a job ID from a previous run (e.g., stored in CI state), you can attach to it without re-running the test:
@@ -205,6 +381,12 @@ Test an entire site using glob patterns.
205
381
  npx @regressionbot/sdk https://example.com --project my-project --scan "/**" --exclude "/admin/**" --concurrency 20
206
382
  ```
207
383
 
384
+ Use `--mask` to hide elements by selector, or `--custom-css` to inject CSS before every
385
+ screenshot:
386
+ ```bash
387
+ npx @regressionbot/sdk https://example.com --project my-site --custom-css "#chat-widget { display: none !important; }"
388
+ ```
389
+
208
390
  #### 3. Job Summary
209
391
  Get detailed results and diff URLs for a completed job.
210
392
  ```bash
package/dist/cli.js CHANGED
@@ -37,6 +37,23 @@ Object.defineProperty(exports, "__esModule", { value: true });
37
37
  const index_1 = require("./index");
38
38
  const security_1 = require("./security");
39
39
  const path = __importStar(require("path"));
40
+ function formatSummary(items) {
41
+ if (!items || items.length === 0)
42
+ return '';
43
+ return items
44
+ .map(item => (item.label ? `[${item.label}] ${item.text}` : item.text))
45
+ .join('\n ');
46
+ }
47
+ function printRegression(r) {
48
+ console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
49
+ console.log(` Diff: ${r.diffUrl}`);
50
+ if (r.verdict) {
51
+ console.log(` Verdict: ${r.verdict.decision} (confidence ${r.verdict.minConfidence.toFixed(2)}-${r.verdict.avgConfidence.toFixed(2)})`);
52
+ }
53
+ const summary = formatSummary(r.regressionbotSummary);
54
+ if (summary)
55
+ console.log(` Summary: ${summary}`);
56
+ }
40
57
  function parseArgs(args) {
41
58
  const options = Object.create(null);
42
59
  options._ = [];
@@ -129,6 +146,7 @@ Options for <url>:
129
146
  --concurrency <n> Max concurrent workers (default 10).
130
147
  --auto-approve Automatically approve results as new baselines.
131
148
  --mask <selectors> Comma-separated CSS selectors to hide (e.g. ".ad,#popup").
149
+ --custom-css <css> CSS injected before each screenshot (max 4096 chars).
132
150
  --skip-summaries Skip waiting for parallel AI summaries in the CLI.
133
151
 
134
152
  Environment Variables:
@@ -167,6 +185,9 @@ async function startJob(url, options) {
167
185
  const selectors = options.mask.split(',').map((s) => s.trim());
168
186
  builder.mask(selectors);
169
187
  }
188
+ if (typeof options['custom-css'] === 'string') {
189
+ builder.customCss(options['custom-css']);
190
+ }
170
191
  const job = await builder.run();
171
192
  console.log(`✅ Job started! ID: ${job.jobId}`);
172
193
  console.log(`📊 Project: ${projectId}`);
@@ -179,7 +200,7 @@ Waiting for completion...
179
200
  `);
180
201
  const result = await job.waitForCompletion(2000, (status) => {
181
202
  const progress = status.progress || { percent: '0' };
182
- const aiStatusDetail = status.summaryStatus && status.summaryStatus !== 'NONE' ? ` [RegressionBot Summary: ${status.summaryStatus}]` : '';
203
+ const aiStatusDetail = status.summaryStatus ? ` [RegressionBot Summary: ${status.summaryStatus}]` : '';
183
204
  process.stdout.write(`\r Status: ${status.status} (${progress.percent}%)${aiStatusDetail}`);
184
205
  }, { waitForSummaries: !options['skip-summaries'] });
185
206
  console.log('\n\n✅ Job Completed.');
@@ -197,12 +218,7 @@ Waiting for completion...
197
218
  }
198
219
  if (summary.regressionCount > 0) {
199
220
  console.log('\n❌ Regressions found:');
200
- summary.regressions.forEach((r) => {
201
- console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
202
- console.log(` Diff: ${r.diffUrl}`);
203
- if (r.regressionbotSummary)
204
- console.log(` Summary: ${r.regressionbotSummary}`);
205
- });
221
+ summary.regressions.forEach(printRegression);
206
222
  console.log(`\nTo approve these changes, run:\n npx regressionbot approve ${job.jobId}`);
207
223
  process.exit(1);
208
224
  }
@@ -237,12 +253,12 @@ Errors: ${summary.errorCount}
237
253
  `);
238
254
  if (summary.regressionCount > 0) {
239
255
  console.log('❌ Regressions found:');
240
- for (const r of summary.regressions) {
241
- console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
242
- console.log(` Diff: ${r.diffUrl}`);
243
- if (r.regressionbotSummary)
244
- console.log(` Summary: ${r.regressionbotSummary}`);
245
- }
256
+ summary.regressions.forEach(printRegression);
257
+ }
258
+ if (summary.intentAssessment) {
259
+ const ia = summary.intentAssessment;
260
+ console.log(`\n🧭 Intent: ${ia.summary}`);
261
+ console.log(` bugs: ${ia.bugCount}, intentional: ${ia.intentionalCount}, noise: ${ia.noiseCount}, needs review: ${ia.needsReviewCount}`);
246
262
  }
247
263
  const doDownload = options.download || options['download-full'];
248
264
  if (doDownload) {
package/dist/index.d.ts CHANGED
@@ -1,7 +1,7 @@
1
- import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary } from './types';
1
+ import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult } from './types';
2
2
  import { sanitizeFilename, sanitizeUrlToPath } from './security';
3
3
  export { sanitizeFilename, sanitizeUrlToPath, Viewports };
4
- export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary };
4
+ export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult };
5
5
  export declare class RegressionBot {
6
6
  private apiKey;
7
7
  private apiUrl;
@@ -40,9 +40,20 @@ export declare class RegressionBot {
40
40
  options?: any;
41
41
  }>;
42
42
  masks?: string[];
43
+ /** CSS injected before screenshotting. Max 4096 characters. */
44
+ customCss?: string;
43
45
  concurrency?: number;
44
46
  autoApprove?: boolean;
47
+ /** What this run is testing, used to judge whether changes were intended. */
48
+ runContext?: RunContext;
45
49
  }): Promise<JobHandle>;
50
+ /**
51
+ * Update the configuration settings for a named project.
52
+ *
53
+ * Changing anything that decides what a capture looks like invalidates the
54
+ * stored baselines — see {@link ProjectConfigUpdate}.
55
+ */
56
+ updateProject(projectName: string, config: ProjectConfigUpdate): Promise<ProjectConfig>;
46
57
  _request<T>(path: string, method?: string, body?: any): Promise<T>;
47
58
  }
48
59
  export declare class JobBuilder {
@@ -70,6 +81,17 @@ export declare class JobBuilder {
70
81
  concurrency(n: number): this;
71
82
  autoApprove(val?: boolean): this;
72
83
  mask(selectors: string[]): this;
84
+ /**
85
+ * Inject custom CSS before each screenshot, e.g. to hide a dynamic widget:
86
+ * `'#chat-widget { display: none !important; }'`. Max 4096 characters.
87
+ */
88
+ customCss(css: string): this;
89
+ /**
90
+ * Describe what this run is testing — commit, PR, expected changes — so
91
+ * RegressionBot can judge whether each change was intentional.
92
+ * Merges with anything set by an earlier call.
93
+ */
94
+ withContext(context: RunContext): this;
73
95
  /**
74
96
  * Compiles the manifest and triggers the API.
75
97
  */
@@ -81,12 +103,7 @@ export declare class JobHandle {
81
103
  constructor(sdk: RegressionBot, jobId: string);
82
104
  getStatus(): Promise<JobStatus>;
83
105
  getSummary(): Promise<JobSummary>;
84
- approve(): Promise<{
85
- message: string;
86
- jobId: string;
87
- approvedUrlsCount: number;
88
- failedCount?: number;
89
- }>;
106
+ approve(): Promise<ApproveResult>;
90
107
  /**
91
108
  * Generate AI summaries on-demand for all regressions in a completed job.
92
109
  */
package/dist/index.js CHANGED
@@ -53,6 +53,17 @@ class RegressionBot {
53
53
  const res = await this._request(`/project/${encodeURIComponent(projectName)}/run`, 'POST', options);
54
54
  return new JobHandle(this, res.jobId);
55
55
  }
56
+ /**
57
+ * Update the configuration settings for a named project.
58
+ *
59
+ * Changing anything that decides what a capture looks like invalidates the
60
+ * stored baselines — see {@link ProjectConfigUpdate}.
61
+ */
62
+ async updateProject(projectName, config) {
63
+ // The API wraps the updated project in { message, project }.
64
+ const res = await this._request(`/project/${encodeURIComponent(projectName)}`, 'PUT', config);
65
+ return res.project;
66
+ }
56
67
  // Internal fetch wrapper
57
68
  async _request(path, method = 'GET', body) {
58
69
  const headers = {
@@ -134,6 +145,23 @@ class JobBuilder {
134
145
  this.manifest.masks = selectors;
135
146
  return this;
136
147
  }
148
+ /**
149
+ * Inject custom CSS before each screenshot, e.g. to hide a dynamic widget:
150
+ * `'#chat-widget { display: none !important; }'`. Max 4096 characters.
151
+ */
152
+ customCss(css) {
153
+ this.manifest.customCss = css;
154
+ return this;
155
+ }
156
+ /**
157
+ * Describe what this run is testing — commit, PR, expected changes — so
158
+ * RegressionBot can judge whether each change was intentional.
159
+ * Merges with anything set by an earlier call.
160
+ */
161
+ withContext(context) {
162
+ this.manifest.runContext = { ...this.manifest.runContext, ...context };
163
+ return this;
164
+ }
137
165
  /**
138
166
  * Compiles the manifest and triggers the API.
139
167
  */
@@ -155,7 +183,9 @@ class JobBuilder {
155
183
  scans: this.manifest.scans,
156
184
  concurrency: this.manifest.concurrency,
157
185
  autoApprove: this.manifest.autoApprove,
158
- masks: this.manifest.masks
186
+ masks: this.manifest.masks,
187
+ customCss: this.manifest.customCss,
188
+ runContext: this.manifest.runContext
159
189
  };
160
190
  const res = await this.sdk._request('/crawl', 'POST', payload);
161
191
  return new JobHandle(this.sdk, res.jobId);
@@ -197,22 +227,29 @@ class JobHandle {
197
227
  const jobDir = path.join(baseDir, safeJobId);
198
228
  if (!fs.existsSync(jobDir))
199
229
  fs.mkdirSync(jobDir, { recursive: true });
200
- const download = async (url, name) => {
230
+ const download = async (url, baseName) => {
201
231
  try {
202
232
  // 🛡️ SECURITY: Prevent SSRF and local file reads by enforcing HTTP(S) protocol
203
233
  (0, security_1.validateProtocol)(url, 'download URL');
204
234
  // 🛡️ SECURITY: Fetch with timeout
205
235
  const res = await (0, security_1.fetchWithTimeout)(url);
206
236
  if (!res.ok) {
207
- console.warn(`Warning: Failed to download ${name} from ${url} (Status: ${res.status})`);
237
+ console.warn(`Warning: Failed to download ${baseName} from ${url} (Status: ${res.status})`);
208
238
  return;
209
239
  }
240
+ // Determine file extension from Content-Type header
241
+ const contentType = res.headers.get('content-type') || '';
242
+ let ext = '.png'; // default fallback
243
+ if (contentType.includes('image/jpeg') || contentType.includes('image/jpg')) {
244
+ ext = '.jpg';
245
+ }
246
+ const finalName = `${baseName}${ext}`;
210
247
  const buffer = Buffer.from(await res.arrayBuffer());
211
- const filePath = path.join(jobDir, name);
248
+ const filePath = path.join(jobDir, finalName);
212
249
  fs.writeFileSync(filePath, buffer);
213
250
  }
214
251
  catch (err) {
215
- console.warn(`Warning: Failed to download ${name}: ${err.message}`);
252
+ console.warn(`Warning: Failed to download ${baseName}: ${err.message}`);
216
253
  }
217
254
  };
218
255
  // Regressions
@@ -220,12 +257,12 @@ class JobHandle {
220
257
  const nameBase = (0, security_1.sanitizeUrlToPath)(r.url);
221
258
  const safeVariant = (0, security_1.sanitizeFilename)(r.variantName);
222
259
  if (r.diffUrl)
223
- await download(r.diffUrl, `${nameBase}_diff_${safeVariant}.png`);
260
+ await download(r.diffUrl, `${nameBase}_diff_${safeVariant}`);
224
261
  if (options.full) {
225
262
  if (r.baselineUrl)
226
- await download(r.baselineUrl, `${nameBase}_baseline_${safeVariant}.png`);
263
+ await download(r.baselineUrl, `${nameBase}_baseline_${safeVariant}`);
227
264
  if (r.currentUrl)
228
- await download(r.currentUrl, `${nameBase}_current_${safeVariant}.png`);
265
+ await download(r.currentUrl, `${nameBase}_current_${safeVariant}`);
229
266
  }
230
267
  }
231
268
  // Matches (Full only)
@@ -234,9 +271,9 @@ class JobHandle {
234
271
  const nameBase = (0, security_1.sanitizeUrlToPath)(m.url);
235
272
  const safeVariant = (0, security_1.sanitizeFilename)(m.variantName);
236
273
  if (m.baselineUrl)
237
- await download(m.baselineUrl, `${nameBase}_baseline_${safeVariant}.png`);
274
+ await download(m.baselineUrl, `${nameBase}_baseline_${safeVariant}`);
238
275
  if (m.currentUrl)
239
- await download(m.currentUrl, `${nameBase}_current_${safeVariant}.png`);
276
+ await download(m.currentUrl, `${nameBase}_current_${safeVariant}`);
240
277
  }
241
278
  }
242
279
  }
package/dist/types.d.ts CHANGED
@@ -9,23 +9,137 @@ export interface ProjectScan {
9
9
  exclude?: string[];
10
10
  };
11
11
  }
12
+ /**
13
+ * Credentials for reaching an origin behind an environment gate — basic auth,
14
+ * a bypass header, or a session cookie. Write-only: the API never returns these,
15
+ * only a `{ configured: true }` presence flag.
16
+ */
17
+ export interface EnvGate {
18
+ basic?: {
19
+ username: string;
20
+ password: string;
21
+ };
22
+ headers?: Record<string, string>;
23
+ cookies?: Array<{
24
+ name: string;
25
+ value: string;
26
+ }>;
27
+ }
28
+ /** How a project's baseline advances between runs. Absent means `approved`. */
29
+ export type BaselinePolicy = 'approved' | 'rolling';
30
+ /** Frequency at which a project runs unattended. */
31
+ export type ProjectSchedule = 'hourly' | 'daily' | 'weekly';
32
+ /**
33
+ * A saved project as the API returns it.
34
+ *
35
+ * This is a read shape. To change a project, pass {@link ProjectConfigUpdate}
36
+ * to updateProject() — the two differ, because gate credentials are written as
37
+ * an {@link EnvGate} but read back only as a presence flag.
38
+ */
12
39
  export interface ProjectConfig {
13
- orgId?: string;
14
40
  name: string;
15
- testOrigin?: string;
41
+ testOrigin: string;
16
42
  baseOrigin?: string;
17
43
  sitemapUrl?: string;
18
44
  paths?: ProjectPath[];
19
45
  scans?: ProjectScan[];
20
- devices?: string[];
46
+ devices: string[];
21
47
  masks?: string[];
48
+ /** CSS injected before screenshotting, to hide dynamic widgets. Max 4096 characters. */
49
+ customCss?: string;
50
+ /** Presence flag only — the stored credential is never returned. */
51
+ testAuth?: {
52
+ configured: true;
53
+ };
54
+ /** Presence flag only — the stored credential is never returned. */
55
+ baseAuth?: {
56
+ configured: true;
57
+ };
22
58
  concurrency?: number;
23
- createdAt?: string;
24
- updatedAt?: string;
59
+ /** Extra instructions handed to the model that writes change summaries. */
60
+ aiPromptInstructions?: string;
61
+ /** Diff percentage below which a regression is skipped by the AI pass. Defaults to 0.01. */
62
+ aiSummaryThreshold?: number;
63
+ /**
64
+ * `approved` (default): a person accepts a run and its captures become the baseline.
65
+ * `rolling`: every completed managed run advances it, and those runs auto-approve.
66
+ */
67
+ baselinePolicy?: BaselinePolicy;
68
+ /** Set when the project runs unattended. Absent means it only runs when triggered. */
69
+ schedule?: ProjectSchedule;
70
+ /**
71
+ * The UTC hour, 0–23, at which a `daily` or `weekly` project runs — `3` is 03:00 UTC.
72
+ * Absent means the slot anchors to whenever the first sweep picked the project up.
73
+ */
74
+ scheduleHourUtc?: number;
75
+ /**
76
+ * The API key recorded when the schedule was set; every scheduled run is attributed
77
+ * to it. Absent if the schedule was set without an API key, in which case AI
78
+ * summaries are skipped for those runs.
79
+ */
80
+ scheduleKeyId?: string;
81
+ /** When the scheduler last started a run. The next run is due one interval after this. */
82
+ lastScheduledRunAt?: string;
83
+ createdAt: string;
84
+ updatedAt: string;
25
85
  lastRunAt?: string;
26
86
  lastJobId?: string;
87
+ baselineVersion?: string;
27
88
  baselineInvalidatedAt?: string;
28
89
  }
90
+ /**
91
+ * The fields updateProject() accepts.
92
+ *
93
+ * Changing anything that decides what a capture looks like — testOrigin,
94
+ * baseOrigin, sitemapUrl, paths, scans, devices, masks, customCss, or a gate
95
+ * credential — invalidates the stored baselines, and the next run re-captures
96
+ * them instead of comparing. Everything else leaves baselines alone.
97
+ * Re-sending an identical value is not a change.
98
+ */
99
+ export interface ProjectConfigUpdate {
100
+ testOrigin?: string;
101
+ baseOrigin?: string;
102
+ sitemapUrl?: string;
103
+ paths?: ProjectPath[];
104
+ scans?: ProjectScan[];
105
+ devices?: string[];
106
+ masks?: string[];
107
+ /** Max 4096 characters. */
108
+ customCss?: string;
109
+ /** Credentials for the test origin's environment gate. Pass null to clear. */
110
+ testAuth?: EnvGate | null;
111
+ /** Credentials for the base origin's environment gate. Pass null to clear. */
112
+ baseAuth?: EnvGate | null;
113
+ concurrency?: number;
114
+ aiPromptInstructions?: string;
115
+ aiSummaryThreshold?: number;
116
+ baselinePolicy?: BaselinePolicy;
117
+ /**
118
+ * Run this project unattended at this frequency. Requires `baselinePolicy: 'rolling'`
119
+ * on a managed project — the API rejects the combination without it, because on the
120
+ * approved policy an unapproved change is re-reported on every subsequent run.
121
+ * Live-vs-live projects store no baseline and are exempt. Pass null to remove.
122
+ *
123
+ * Without `scheduleHourUtc` the first run starts at the next hourly sweep and the
124
+ * cadence anchors to it.
125
+ */
126
+ schedule?: ProjectSchedule | null;
127
+ /**
128
+ * Pin a `daily` or `weekly` schedule to this UTC hour, 0–23 — `3` means 03:00 UTC.
129
+ * The first run waits for the hour too, rather than firing immediately and settling
130
+ * into the hour from the following day. Hours only, because the scheduler sweeps once
131
+ * an hour; UTC only, because RegressionBot has no timezone to convert from.
132
+ *
133
+ * Rejected on an `hourly` schedule, which already runs every hour, and rejected if the
134
+ * project has no schedule to apply it to — either one already set, or one sent in the
135
+ * same update. Clearing the schedule with
136
+ * `schedule: null` clears this too, so re-enabling later cannot resurrect an hour
137
+ * nobody asked for on that run. Pass null to clear it on its own.
138
+ *
139
+ * A missed slot waits for the next one rather than catching up.
140
+ */
141
+ scheduleHourUtc?: number | null;
142
+ }
29
143
  export interface VRConfig {
30
144
  apiKey?: string;
31
145
  apiUrl?: string;
@@ -52,53 +166,105 @@ export declare const Viewports: {
52
166
  readonly height: 667;
53
167
  };
54
168
  };
169
+ export type VerdictDecision = 'intentional' | 'bug' | 'noise' | 'needs_review';
170
+ /** How one labelled region of a diff was judged against the run's stated intent. */
171
+ export interface RegionVerdict {
172
+ decision: VerdictDecision;
173
+ /** 0–1. */
174
+ confidence: number;
175
+ reasoning: string;
176
+ }
177
+ /** The rolled-up verdict for one result, plus the per-region verdicts behind it. */
178
+ export interface ResultVerdict {
179
+ decision: VerdictDecision;
180
+ minConfidence: number;
181
+ avgConfidence: number;
182
+ /** Keyed by region label (A, B, C…). Empty when the verdict came from the text-only pass. */
183
+ regions: Record<string, RegionVerdict>;
184
+ }
185
+ export interface RegressionbotSummaryItem {
186
+ /** Region letter (A, B, C…), or an empty string for a whole-page change. */
187
+ label: string;
188
+ /** Single-sentence description of what changed in that region. */
189
+ text: string;
190
+ verdict?: RegionVerdict;
191
+ }
55
192
  /**
56
193
  * Result for a single URL + device comparison.
57
194
  *
58
- * Returned by both getStatus() and getSummary(). Pre-signed image URLs
59
- * (baselineUrl, currentUrl, diffUrl) are only populated by getSummary() —
60
- * they are absent when reading results from getStatus().
195
+ * Returned by getStatus() and, split across `regressions`/`matches`, by
196
+ * getSummary(). Both endpoints pre-sign the image URLs; a URL is null when
197
+ * that image does not exist for this result (no baseline yet, no diff, …).
61
198
  */
62
199
  export interface PageResult {
63
200
  url: string;
64
- /** Device variant tested (e.g. "desktop_chrome", "iphone_12"). */
201
+ /** Whether the capture and comparison succeeded. */
202
+ status: 'SUCCESS' | 'ERROR';
203
+ /** Device variant tested (e.g. "Desktop Chrome", "iPhone 12"). */
65
204
  variantName: string;
66
205
  /** Percentage of pixels that differed from the baseline (0 = identical). */
67
206
  diffPercentage: number;
68
- /** Raw count of differing pixels. */
69
- diffCount: number;
70
207
  /** Perceptual similarity 0-100 (SSIM). */
71
208
  visualMatchScore: number;
209
+ /** True when this capture became a baseline because none existed to compare against. */
210
+ isNewBaseline: boolean;
211
+ /** Why the capture or comparison failed. Present when status is ERROR. */
212
+ errorMessage?: string;
72
213
  /** Plain-English description of what visually changed, generated by RegressionBot. */
73
- regressionbotSummary?: string;
214
+ regressionbotSummary?: RegressionbotSummaryItem[];
215
+ /**
216
+ * Whether the change was judged intentional, a bug, or noise. Only produced when
217
+ * the run carried a {@link RunContext}, and only once summaryStatus is COMPLETE.
218
+ */
219
+ verdict?: ResultVerdict;
220
+ /** Selectors of the elements that changed, when the DOM comparison could identify them. */
221
+ elementsChanged?: string[];
222
+ /**
223
+ * Set when the DOM comparison could not run. The result is a plain pixel diff with
224
+ * no semantic change types, so treat its classifications as absent rather than as
225
+ * "nothing structural changed".
226
+ */
227
+ domAssistSkipReason?: string;
74
228
  /** Pre-signed URL for the stored baseline screenshot. */
75
- baselineUrl?: string;
229
+ baselineUrl: string | null;
76
230
  /** Pre-signed URL for the newly captured screenshot. */
77
- currentUrl?: string;
231
+ currentUrl: string | null;
78
232
  /** Pre-signed URL for the side-by-side annotated diff image. */
79
- diffUrl?: string;
233
+ diffUrl: string | null;
234
+ /**
235
+ * Pre-signed URL for the diff mask. getStatus() always sends the key, null when
236
+ * there is no mask; getSummary() omits it entirely in that case.
237
+ */
238
+ maskUrl?: string | null;
80
239
  }
81
240
  export interface JobProgress {
82
241
  total: number;
83
242
  completed: number;
84
243
  percent: string;
85
244
  }
245
+ /** Progress of the asynchronous AI summary pass. */
246
+ export type SummaryStatus = 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'FAILED';
86
247
  export interface JobStatus {
87
248
  jobId: string;
88
249
  status: 'INITIALIZING' | 'PROCESSING' | 'FINISHING' | 'SUMMARIZING' | 'COMPLETED' | 'APPROVED' | 'FAILED';
89
- summaryStatus?: 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'NONE';
90
- error?: string;
91
- progress?: JobProgress;
92
- executionTime?: number;
93
- createdAt?: string;
94
- /** Partial or complete results. Present once workers start completing. Pre-signed image URLs are not included — call getSummary() for those. */
95
- results?: PageResult[];
250
+ /**
251
+ * PENDING means not yet started, COMPLETE that regressionbotSummary is populated
252
+ * (or that nothing needed one). Poll until COMPLETE before reading summaries.
253
+ */
254
+ summaryStatus: SummaryStatus;
255
+ error: string | null;
256
+ progress: JobProgress;
257
+ executionTime: number;
258
+ createdAt: string;
259
+ /** Partial or complete results. Fills in as workers finish. */
260
+ results: PageResult[];
96
261
  }
97
262
  export interface JobSummary {
98
263
  jobId: string;
99
- status: string;
100
- summaryStatus?: 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'NONE';
101
- error?: string | null;
264
+ /** getSummary() throws until the job reaches one of these. */
265
+ status: 'COMPLETED' | 'APPROVED' | 'FAILED';
266
+ summaryStatus: SummaryStatus;
267
+ error: string | null;
102
268
  totalUrls: number;
103
269
  completedCount: number;
104
270
  /** Overall quality score 0–100 across all tested pages. */
@@ -108,9 +274,9 @@ export interface JobSummary {
108
274
  matchCount: number;
109
275
  newBaselineCount: number;
110
276
  errorCount: number;
111
- /** Pages where a visual regression was detected. Includes pre-signed image URLs. */
277
+ /** Pages where a visual regression was detected. */
112
278
  regressions: PageResult[];
113
- /** Pages that matched the baseline. Includes pre-signed image URLs. */
279
+ /** Pages that matched the baseline. */
114
280
  matches: PageResult[];
115
281
  /** Pages captured for the first time — no baseline existed to compare against. */
116
282
  newBaselines: Array<{
@@ -123,6 +289,49 @@ export interface JobSummary {
123
289
  variantName: string;
124
290
  errorMessage: string;
125
291
  }>;
292
+ /** Whole-job roll-up of how the changes line up with the run's stated intent. */
293
+ intentAssessment: IntentAssessment;
294
+ /** The intent this run was given, echoed back. */
295
+ runContext?: RunContext;
296
+ }
297
+ /**
298
+ * What a run is testing. Supplying it lets RegressionBot judge each change as
299
+ * intentional, a bug, or noise, rather than just reporting that pixels moved.
300
+ */
301
+ export interface RunContext {
302
+ /** Short description of what this run is testing. */
303
+ changeDescription?: string;
304
+ gitCommitSha?: string;
305
+ /** Git commit subject line. */
306
+ gitCommitMessage?: string;
307
+ prTitle?: string;
308
+ /** Markdown. Truncated to 2000 characters by the API. */
309
+ prDescription?: string;
310
+ /** Explicit list of expected visual changes. */
311
+ expectedChanges?: string[];
312
+ /** Files or component names changed. */
313
+ scope?: string[];
314
+ }
315
+ export interface IntentAssessment {
316
+ /** False when the run carried no RunContext, in which case the counts are all zero. */
317
+ intentProvided: boolean;
318
+ bugCount: number;
319
+ intentionalCount: number;
320
+ noiseCount: number;
321
+ needsReviewCount: number;
322
+ /** True when every regression is accounted for by the stated intent. */
323
+ allAccountedFor: boolean;
324
+ /** One line summarising the job, safe to print straight into CI output. */
325
+ summary: string;
326
+ }
327
+ export interface ApproveResult {
328
+ message: string;
329
+ jobId: string;
330
+ approvedUrlsCount: number;
331
+ /** Present when some captures could not be promoted. */
332
+ failedCount?: number;
333
+ /** Pages skipped because another job updated their baseline first. */
334
+ conflictedUrls?: string[];
126
335
  }
127
336
  export interface JobAiSummary {
128
337
  message: string;
@@ -130,6 +339,6 @@ export interface JobAiSummary {
130
339
  summaries: Array<{
131
340
  url: string;
132
341
  variantName: string;
133
- regressionbotSummary: string;
342
+ regressionbotSummary: RegressionbotSummaryItem[];
134
343
  }>;
135
344
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@regressionbot/sdk",
3
- "version": "1.2.2",
3
+ "version": "2.0.0",
4
4
  "description": "The official SDK for regressionbot.com - the simplest way to automate visual regression testing.",
5
5
  "homepage": "https://regressionbot.com",
6
6
  "repository": {