@regressionbot/sdk 1.2.3 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -13,7 +13,7 @@ RegressionBot is a declarative visual regression testing platform that helps you
13
13
 
14
14
  Unlike traditional visual diffing libraries, RegressionBot is designed for modern, automated development loops and agentic pipelines:
15
15
 
16
- - **Highly Accurate Regressions (Zero Noise)**: Leveraging advanced pixel-matching algorithms and element masking (using CSS selectors or the automatic `data-vr-mask` attribute), RegressionBot eliminates false positives caused by dynamic data, layout shifting, or third-party widgets.
16
+ - **Highly Accurate Regressions (Less Noise)**: Leveraging advanced pixel-matching algorithms and element masking (using CSS selectors or the automatic `data-vr-mask` attribute), RegressionBot minimizes false positives caused by dynamic data, layout shifting, or third-party widgets.
17
17
  - **Plain-English Summaries**: No more manual screenshot comparisons. RegressionBot translates visual diffs into concise, plain-English descriptions of what changed, so you know exactly what was modified at a glance.
18
18
  - **Agentic Workflow Ready**: Built from the ground up to support autonomous coding agents (like Gemini or Claude) and automated developer loops. Through standard API endpoints, CLI commands, and Model Context Protocol (MCP) integrations, agents can trigger tests, read plain-English results, and approve baseline changes programmatically without human intervention.
19
19
 
@@ -24,6 +24,8 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
24
24
  - **Matrix Testing**: Test multiple devices and viewports in a single job.
25
25
  - **Auto-Discovery**: Scan sitemaps with glob patterns and limits.
26
26
  - **RegressionBot Summaries**: Plain-English change descriptions for every regression, generated on-demand via the API.
27
+ - **Intent-Aware Verdicts**: Describe what a run is meant to change, and every regression comes back judged as intentional, a bug, or noise.
28
+ - **Scheduled Checks**: Run a saved project unattended, hourly, daily, or weekly — and at a set UTC hour.
27
29
  - **Project-Based Baselines**: Save and reuse test configurations; share visual history across environments.
28
30
  - **Auto-Approval**: Automatically promote screenshots to baselines on jobs that pass your criteria.
29
31
  - **Zero Infrastructure**: No browser maintenance or server provisioning — RegressionBot handles it all.
@@ -34,6 +36,44 @@ Unlike traditional visual diffing libraries, RegressionBot is designed for moder
34
36
  npm install @regressionbot/sdk
35
37
  ```
36
38
 
39
+ ## Migration from 1.x
40
+
41
+ 2.0.0 re-types the SDK against what the API actually returns, taken from the
42
+ orchestrator's implementation rather than from a spec that had drifted in both
43
+ directions. Most of the breakage is caught by the compiler. One is not:
44
+
45
+ ```typescript
46
+ // 1.x — updateProject() returned the raw { message, project } envelope,
47
+ // so every field read off it was undefined at runtime
48
+ const project = await rb.updateProject('site', { schedule: 'daily' });
49
+ project.project.schedule; // this is where the value actually was
50
+
51
+ // 2.0 — the envelope is unwrapped for you
52
+ const project = await rb.updateProject('site', { schedule: 'daily' });
53
+ project.schedule; // 'daily'
54
+ ```
55
+
56
+ The rest surface as type errors:
57
+
58
+ - **`PageResult.diffCount` is gone.** No endpoint ever returned it — both
59
+ `getJobStatus` and `getJobSummary` strip it. `PageResult` gains the fields they
60
+ do return: `status`, `isNewBaseline`, `errorMessage`, `elementsChanged`,
61
+ `domAssistSkipReason`, `maskUrl` and `verdict`. Image URLs are typed nullable,
62
+ which they always were.
63
+ - **`updateProject()` takes `ProjectConfigUpdate`, not `Partial<ProjectConfig>`.**
64
+ The read and write shapes genuinely differ: a gate credential is written as an
65
+ `EnvGate` and read back only as `{ configured: true }`.
66
+ - **`ProjectConfig.orgId` is gone** — the API stopped returning it.
67
+ - **`summaryStatus` can be `FAILED`**, which the old union omitted. `error`,
68
+ `progress`, `results`, `createdAt` and `intentAssessment` are always returned.
69
+ - **`approve()` gains `conflictedUrls`**, returned when another job updated a
70
+ baseline first.
71
+
72
+ New in 2.0 and requiring a current API: `.customCss()` and `.withContext()` on
73
+ the builder, intent-aware verdicts, environment gates, baseline policies, and
74
+ scheduling. `scheduleHourUtc` needs API 2.7.0 or later — on an older API the
75
+ field is accepted and ignored, and the schedule stays anchored to its first run.
76
+
37
77
  ## Usage
38
78
 
39
79
  ### Basic Example
@@ -91,6 +131,12 @@ const job = await rb
91
131
  .mask(['.ads', '#modal']) // Manual selectors
92
132
  // Tip: Adding 'data-vr-mask' to your HTML elements masks them automatically!
93
133
 
134
+ // Custom CSS: Injected before every screenshot (max 4096 chars)
135
+ .customCss('#chat-widget { display: none !important; }')
136
+
137
+ // Intent: What this run is testing, so changes can be judged against it
138
+ .withContext({ changeDescription: 'Restyle the pricing table' })
139
+
94
140
  // Execute: Compiles manifest and triggers the API
95
141
  .run();
96
142
 
@@ -112,9 +158,9 @@ const summary = await job.getSummary();
112
158
  if (summary.regressions.length > 0) {
113
159
  console.log(`\n${summary.regressions.length} regressions found:`);
114
160
  for (const regression of summary.regressions) {
115
- if (regression.regressionbotSummary) {
116
- console.log(`\nRegressionBot Summary for ${regression.url}:`);
117
- console.log(`> ${regression.regressionbotSummary}`);
161
+ for (const item of regression.regressionbotSummary ?? []) {
162
+ // `label` is the region letter (A, B, C…), or '' for a whole-page change
163
+ console.log(`[${item.label}] ${item.text}`);
118
164
  }
119
165
  }
120
166
  }
@@ -124,6 +170,68 @@ const aiResult = await job.generateAiSummary();
124
170
  console.log(`Generated summaries for ${aiResult.summaries.length} regressions.`);
125
171
  ```
126
172
 
173
+ ### Intent-Aware Verdicts
174
+
175
+ Tell RegressionBot what a run is meant to change, and every regression comes back judged
176
+ against that intent — `intentional`, `bug`, `noise`, or `needs_review`.
177
+
178
+ ```typescript
179
+ const job = await rb
180
+ .test(process.env.VERCEL_PREVIEW_URL)
181
+ .forProject('marketing-site-v2')
182
+ .withContext({
183
+ changeDescription: 'Restyle the pricing table',
184
+ gitCommitSha: process.env.GITHUB_SHA,
185
+ prTitle: process.env.PR_TITLE,
186
+ expectedChanges: ['Pricing table background is now dark'],
187
+ scope: ['components/PricingTable.tsx'],
188
+ })
189
+ .run();
190
+
191
+ await job.waitForCompletion(2000, undefined, { waitForSummaries: true });
192
+ const summary = await job.getSummary();
193
+
194
+ // Whole-job roll-up: does everything that changed line up with the stated intent?
195
+ console.log(summary.intentAssessment?.summary);
196
+ console.log(`Bugs: ${summary.intentAssessment?.bugCount}`);
197
+
198
+ // Per-result verdict, plus the per-region verdicts behind it
199
+ for (const r of summary.regressions) {
200
+ console.log(`${r.url}: ${r.verdict?.decision} (${r.verdict?.avgConfidence})`);
201
+ }
202
+
203
+ // Fail CI only on changes that were not intended
204
+ const bugs = summary.regressions.filter(r => r.verdict?.decision === 'bug');
205
+ if (bugs.length > 0) process.exit(1);
206
+ ```
207
+
208
+ `runProject()` takes the same context, so a run from a saved project can be
209
+ intent-aware too:
210
+
211
+ ```typescript
212
+ const job = await rb.runProject('marketing-site-v2', {
213
+ testOrigin: process.env.VERCEL_PREVIEW_URL,
214
+ runContext: { changeDescription: 'Restyle the pricing table' },
215
+ });
216
+ ```
217
+
218
+ ### Hiding Dynamic Content
219
+
220
+ `.mask()` hides elements by selector. `.customCss()` injects arbitrary CSS before each
221
+ screenshot when masking isn't enough — collapsing an animation, pinning a carousel, or
222
+ neutralising a third-party widget. Max 4096 characters.
223
+
224
+ ```typescript
225
+ const job = await rb
226
+ .test('https://preview.myapp.com')
227
+ .forProject('my-app-web')
228
+ .customCss(`
229
+ #chat-widget { display: none !important; }
230
+ * { animation: none !important; transition: none !important; }
231
+ `)
232
+ .run();
233
+ ```
234
+
127
235
  ### Progress Tracking
128
236
 
129
237
  Pass a callback to `waitForCompletion` to receive status updates while the job runs:
@@ -177,6 +285,69 @@ const summary = await job.getSummary();
177
285
  console.log(`Score: ${summary.overallScore}/100`);
178
286
  ```
179
287
 
288
+ ### Environment Gates
289
+
290
+ If an origin sits behind basic auth, a bypass header, or a session cookie, store the
291
+ credential on the project. It is encrypted at rest and never returned — reads report
292
+ only `{ configured: true }`.
293
+
294
+ ```typescript
295
+ await rb.updateProject('marketing-site-v2', {
296
+ testAuth: { basic: { username: 'preview', password: process.env.PREVIEW_PASSWORD! } },
297
+ // or: { headers: { 'x-vercel-protection-bypass': token } }
298
+ // or: { cookies: [{ name: 'session', value: token }] }
299
+ });
300
+
301
+ const project = await rb.getProject('marketing-site-v2');
302
+ console.log(project.testAuth); // { configured: true }
303
+
304
+ // Clear it
305
+ await rb.updateProject('marketing-site-v2', { testAuth: null });
306
+ ```
307
+
308
+ Setting or clearing a gate credential invalidates the stored baselines, since it changes
309
+ what the capture can reach.
310
+
311
+ ### Scheduled Checks
312
+
313
+ A project can run unattended on a fixed cadence. Scheduling requires
314
+ `baselinePolicy: 'rolling'` on a managed project — on the default `approved` policy an
315
+ unapproved change would be re-reported on every subsequent run, so the API rejects the
316
+ combination. Live-vs-live projects store no baseline and are exempt.
317
+
318
+ ```typescript
319
+ // Turn on a daily unattended check
320
+ await rb.updateProject('marketing-site-v2', {
321
+ baselinePolicy: 'rolling',
322
+ schedule: 'daily', // 'hourly' | 'daily' | 'weekly'
323
+ });
324
+
325
+ // Or pin it to a UTC hour — this one runs at 03:00 UTC every day
326
+ await rb.updateProject('marketing-site-v2', {
327
+ baselinePolicy: 'rolling',
328
+ schedule: 'daily',
329
+ scheduleHourUtc: 3, // 0–23, UTC
330
+ });
331
+
332
+ // Turn it back off (this clears scheduleHourUtc too)
333
+ await rb.updateProject('marketing-site-v2', { schedule: null });
334
+
335
+ // Read back when the scheduler last fired — the next run is due one interval after this
336
+ const project = await rb.getProject('marketing-site-v2');
337
+ console.log(project.schedule, project.scheduleHourUtc, project.lastScheduledRunAt);
338
+ ```
339
+
340
+ Without `scheduleHourUtc` the first run starts at the next hourly sweep and the cadence
341
+ anchors to it. With it, the first run waits for that hour as well. Hours only, since the
342
+ scheduler sweeps once an hour, and UTC only. It is rejected on an `hourly` schedule, and
343
+ rejected on a project with no schedule to apply it to. Clearing the schedule clears the
344
+ hour with it, and a missed slot waits for the next one rather than catching up.
345
+
346
+ Setting a schedule, its hour, or a baseline policy does **not** invalidate
347
+ baselines — only changes that affect what a capture looks like (`testOrigin`,
348
+ `baseOrigin`, `sitemapUrl`, `paths`, `scans`, `devices`, `masks`, `customCss`) do that.
349
+ Billing is per comparison, so cost scales with frequency.
350
+
180
351
  ### Reconnecting to an Existing Job
181
352
 
182
353
  If you have a job ID from a previous run (e.g., stored in CI state), you can attach to it without re-running the test:
@@ -210,6 +381,12 @@ Test an entire site using glob patterns.
210
381
  npx @regressionbot/sdk https://example.com --project my-project --scan "/**" --exclude "/admin/**" --concurrency 20
211
382
  ```
212
383
 
384
+ Use `--mask` to hide elements by selector, or `--custom-css` to inject CSS before every
385
+ screenshot:
386
+ ```bash
387
+ npx @regressionbot/sdk https://example.com --project my-site --custom-css "#chat-widget { display: none !important; }"
388
+ ```
389
+
213
390
  #### 3. Job Summary
214
391
  Get detailed results and diff URLs for a completed job.
215
392
  ```bash
package/dist/cli.js CHANGED
@@ -37,6 +37,23 @@ Object.defineProperty(exports, "__esModule", { value: true });
37
37
  const index_1 = require("./index");
38
38
  const security_1 = require("./security");
39
39
  const path = __importStar(require("path"));
40
+ function formatSummary(items) {
41
+ if (!items || items.length === 0)
42
+ return '';
43
+ return items
44
+ .map(item => (item.label ? `[${item.label}] ${item.text}` : item.text))
45
+ .join('\n ');
46
+ }
47
+ function printRegression(r) {
48
+ console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
49
+ console.log(` Diff: ${r.diffUrl}`);
50
+ if (r.verdict) {
51
+ console.log(` Verdict: ${r.verdict.decision} (confidence ${r.verdict.minConfidence.toFixed(2)}-${r.verdict.avgConfidence.toFixed(2)})`);
52
+ }
53
+ const summary = formatSummary(r.regressionbotSummary);
54
+ if (summary)
55
+ console.log(` Summary: ${summary}`);
56
+ }
40
57
  function parseArgs(args) {
41
58
  const options = Object.create(null);
42
59
  options._ = [];
@@ -129,6 +146,7 @@ Options for <url>:
129
146
  --concurrency <n> Max concurrent workers (default 10).
130
147
  --auto-approve Automatically approve results as new baselines.
131
148
  --mask <selectors> Comma-separated CSS selectors to hide (e.g. ".ad,#popup").
149
+ --custom-css <css> CSS injected before each screenshot (max 4096 chars).
132
150
  --skip-summaries Skip waiting for parallel AI summaries in the CLI.
133
151
 
134
152
  Environment Variables:
@@ -167,6 +185,9 @@ async function startJob(url, options) {
167
185
  const selectors = options.mask.split(',').map((s) => s.trim());
168
186
  builder.mask(selectors);
169
187
  }
188
+ if (typeof options['custom-css'] === 'string') {
189
+ builder.customCss(options['custom-css']);
190
+ }
170
191
  const job = await builder.run();
171
192
  console.log(`✅ Job started! ID: ${job.jobId}`);
172
193
  console.log(`📊 Project: ${projectId}`);
@@ -179,7 +200,7 @@ Waiting for completion...
179
200
  `);
180
201
  const result = await job.waitForCompletion(2000, (status) => {
181
202
  const progress = status.progress || { percent: '0' };
182
- const aiStatusDetail = status.summaryStatus && status.summaryStatus !== 'NONE' ? ` [RegressionBot Summary: ${status.summaryStatus}]` : '';
203
+ const aiStatusDetail = status.summaryStatus ? ` [RegressionBot Summary: ${status.summaryStatus}]` : '';
183
204
  process.stdout.write(`\r Status: ${status.status} (${progress.percent}%)${aiStatusDetail}`);
184
205
  }, { waitForSummaries: !options['skip-summaries'] });
185
206
  console.log('\n\n✅ Job Completed.');
@@ -197,12 +218,7 @@ Waiting for completion...
197
218
  }
198
219
  if (summary.regressionCount > 0) {
199
220
  console.log('\n❌ Regressions found:');
200
- summary.regressions.forEach((r) => {
201
- console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
202
- console.log(` Diff: ${r.diffUrl}`);
203
- if (r.regressionbotSummary)
204
- console.log(` Summary: ${r.regressionbotSummary}`);
205
- });
221
+ summary.regressions.forEach(printRegression);
206
222
  console.log(`\nTo approve these changes, run:\n npx regressionbot approve ${job.jobId}`);
207
223
  process.exit(1);
208
224
  }
@@ -237,12 +253,12 @@ Errors: ${summary.errorCount}
237
253
  `);
238
254
  if (summary.regressionCount > 0) {
239
255
  console.log('❌ Regressions found:');
240
- for (const r of summary.regressions) {
241
- console.log(`- ${r.url} [${r.variantName}] (Score: ${r.visualMatchScore.toFixed(2)})`);
242
- console.log(` Diff: ${r.diffUrl}`);
243
- if (r.regressionbotSummary)
244
- console.log(` Summary: ${r.regressionbotSummary}`);
245
- }
256
+ summary.regressions.forEach(printRegression);
257
+ }
258
+ if (summary.intentAssessment) {
259
+ const ia = summary.intentAssessment;
260
+ console.log(`\n🧭 Intent: ${ia.summary}`);
261
+ console.log(` bugs: ${ia.bugCount}, intentional: ${ia.intentionalCount}, noise: ${ia.noiseCount}, needs review: ${ia.needsReviewCount}`);
246
262
  }
247
263
  const doDownload = options.download || options['download-full'];
248
264
  if (doDownload) {
package/dist/index.d.ts CHANGED
@@ -1,7 +1,7 @@
1
- import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary } from './types';
1
+ import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult } from './types';
2
2
  import { sanitizeFilename, sanitizeUrlToPath } from './security';
3
3
  export { sanitizeFilename, sanitizeUrlToPath, Viewports };
4
- export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary };
4
+ export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult };
5
5
  export declare class RegressionBot {
6
6
  private apiKey;
7
7
  private apiUrl;
@@ -40,13 +40,20 @@ export declare class RegressionBot {
40
40
  options?: any;
41
41
  }>;
42
42
  masks?: string[];
43
+ /** CSS injected before screenshotting. Max 4096 characters. */
44
+ customCss?: string;
43
45
  concurrency?: number;
44
46
  autoApprove?: boolean;
47
+ /** What this run is testing, used to judge whether changes were intended. */
48
+ runContext?: RunContext;
45
49
  }): Promise<JobHandle>;
46
50
  /**
47
51
  * Update the configuration settings for a named project.
52
+ *
53
+ * Changing anything that decides what a capture looks like invalidates the
54
+ * stored baselines — see {@link ProjectConfigUpdate}.
48
55
  */
49
- updateProject(projectName: string, config: Partial<ProjectConfig>): Promise<ProjectConfig>;
56
+ updateProject(projectName: string, config: ProjectConfigUpdate): Promise<ProjectConfig>;
50
57
  _request<T>(path: string, method?: string, body?: any): Promise<T>;
51
58
  }
52
59
  export declare class JobBuilder {
@@ -74,6 +81,17 @@ export declare class JobBuilder {
74
81
  concurrency(n: number): this;
75
82
  autoApprove(val?: boolean): this;
76
83
  mask(selectors: string[]): this;
84
+ /**
85
+ * Inject custom CSS before each screenshot, e.g. to hide a dynamic widget:
86
+ * `'#chat-widget { display: none !important; }'`. Max 4096 characters.
87
+ */
88
+ customCss(css: string): this;
89
+ /**
90
+ * Describe what this run is testing — commit, PR, expected changes — so
91
+ * RegressionBot can judge whether each change was intentional.
92
+ * Merges with anything set by an earlier call.
93
+ */
94
+ withContext(context: RunContext): this;
77
95
  /**
78
96
  * Compiles the manifest and triggers the API.
79
97
  */
@@ -85,12 +103,7 @@ export declare class JobHandle {
85
103
  constructor(sdk: RegressionBot, jobId: string);
86
104
  getStatus(): Promise<JobStatus>;
87
105
  getSummary(): Promise<JobSummary>;
88
- approve(): Promise<{
89
- message: string;
90
- jobId: string;
91
- approvedUrlsCount: number;
92
- failedCount?: number;
93
- }>;
106
+ approve(): Promise<ApproveResult>;
94
107
  /**
95
108
  * Generate AI summaries on-demand for all regressions in a completed job.
96
109
  */
package/dist/index.js CHANGED
@@ -55,9 +55,14 @@ class RegressionBot {
55
55
  }
56
56
  /**
57
57
  * Update the configuration settings for a named project.
58
+ *
59
+ * Changing anything that decides what a capture looks like invalidates the
60
+ * stored baselines — see {@link ProjectConfigUpdate}.
58
61
  */
59
62
  async updateProject(projectName, config) {
60
- return this._request(`/project/${encodeURIComponent(projectName)}`, 'PUT', config);
63
+ // The API wraps the updated project in { message, project }.
64
+ const res = await this._request(`/project/${encodeURIComponent(projectName)}`, 'PUT', config);
65
+ return res.project;
61
66
  }
62
67
  // Internal fetch wrapper
63
68
  async _request(path, method = 'GET', body) {
@@ -140,6 +145,23 @@ class JobBuilder {
140
145
  this.manifest.masks = selectors;
141
146
  return this;
142
147
  }
148
+ /**
149
+ * Inject custom CSS before each screenshot, e.g. to hide a dynamic widget:
150
+ * `'#chat-widget { display: none !important; }'`. Max 4096 characters.
151
+ */
152
+ customCss(css) {
153
+ this.manifest.customCss = css;
154
+ return this;
155
+ }
156
+ /**
157
+ * Describe what this run is testing — commit, PR, expected changes — so
158
+ * RegressionBot can judge whether each change was intentional.
159
+ * Merges with anything set by an earlier call.
160
+ */
161
+ withContext(context) {
162
+ this.manifest.runContext = { ...this.manifest.runContext, ...context };
163
+ return this;
164
+ }
143
165
  /**
144
166
  * Compiles the manifest and triggers the API.
145
167
  */
@@ -161,7 +183,9 @@ class JobBuilder {
161
183
  scans: this.manifest.scans,
162
184
  concurrency: this.manifest.concurrency,
163
185
  autoApprove: this.manifest.autoApprove,
164
- masks: this.manifest.masks
186
+ masks: this.manifest.masks,
187
+ customCss: this.manifest.customCss,
188
+ runContext: this.manifest.runContext
165
189
  };
166
190
  const res = await this.sdk._request('/crawl', 'POST', payload);
167
191
  return new JobHandle(this.sdk, res.jobId);
@@ -203,22 +227,29 @@ class JobHandle {
203
227
  const jobDir = path.join(baseDir, safeJobId);
204
228
  if (!fs.existsSync(jobDir))
205
229
  fs.mkdirSync(jobDir, { recursive: true });
206
- const download = async (url, name) => {
230
+ const download = async (url, baseName) => {
207
231
  try {
208
232
  // 🛡️ SECURITY: Prevent SSRF and local file reads by enforcing HTTP(S) protocol
209
233
  (0, security_1.validateProtocol)(url, 'download URL');
210
234
  // 🛡️ SECURITY: Fetch with timeout
211
235
  const res = await (0, security_1.fetchWithTimeout)(url);
212
236
  if (!res.ok) {
213
- console.warn(`Warning: Failed to download ${name} from ${url} (Status: ${res.status})`);
237
+ console.warn(`Warning: Failed to download ${baseName} from ${url} (Status: ${res.status})`);
214
238
  return;
215
239
  }
240
+ // Determine file extension from Content-Type header
241
+ const contentType = res.headers.get('content-type') || '';
242
+ let ext = '.png'; // default fallback
243
+ if (contentType.includes('image/jpeg') || contentType.includes('image/jpg')) {
244
+ ext = '.jpg';
245
+ }
246
+ const finalName = `${baseName}${ext}`;
216
247
  const buffer = Buffer.from(await res.arrayBuffer());
217
- const filePath = path.join(jobDir, name);
248
+ const filePath = path.join(jobDir, finalName);
218
249
  fs.writeFileSync(filePath, buffer);
219
250
  }
220
251
  catch (err) {
221
- console.warn(`Warning: Failed to download ${name}: ${err.message}`);
252
+ console.warn(`Warning: Failed to download ${baseName}: ${err.message}`);
222
253
  }
223
254
  };
224
255
  // Regressions
@@ -226,12 +257,12 @@ class JobHandle {
226
257
  const nameBase = (0, security_1.sanitizeUrlToPath)(r.url);
227
258
  const safeVariant = (0, security_1.sanitizeFilename)(r.variantName);
228
259
  if (r.diffUrl)
229
- await download(r.diffUrl, `${nameBase}_diff_${safeVariant}.png`);
260
+ await download(r.diffUrl, `${nameBase}_diff_${safeVariant}`);
230
261
  if (options.full) {
231
262
  if (r.baselineUrl)
232
- await download(r.baselineUrl, `${nameBase}_baseline_${safeVariant}.png`);
263
+ await download(r.baselineUrl, `${nameBase}_baseline_${safeVariant}`);
233
264
  if (r.currentUrl)
234
- await download(r.currentUrl, `${nameBase}_current_${safeVariant}.png`);
265
+ await download(r.currentUrl, `${nameBase}_current_${safeVariant}`);
235
266
  }
236
267
  }
237
268
  // Matches (Full only)
@@ -240,9 +271,9 @@ class JobHandle {
240
271
  const nameBase = (0, security_1.sanitizeUrlToPath)(m.url);
241
272
  const safeVariant = (0, security_1.sanitizeFilename)(m.variantName);
242
273
  if (m.baselineUrl)
243
- await download(m.baselineUrl, `${nameBase}_baseline_${safeVariant}.png`);
274
+ await download(m.baselineUrl, `${nameBase}_baseline_${safeVariant}`);
244
275
  if (m.currentUrl)
245
- await download(m.currentUrl, `${nameBase}_current_${safeVariant}.png`);
276
+ await download(m.currentUrl, `${nameBase}_current_${safeVariant}`);
246
277
  }
247
278
  }
248
279
  }
package/dist/types.d.ts CHANGED
@@ -9,23 +9,137 @@ export interface ProjectScan {
9
9
  exclude?: string[];
10
10
  };
11
11
  }
12
+ /**
13
+ * Credentials for reaching an origin behind an environment gate — basic auth,
14
+ * a bypass header, or a session cookie. Write-only: the API never returns these,
15
+ * only a `{ configured: true }` presence flag.
16
+ */
17
+ export interface EnvGate {
18
+ basic?: {
19
+ username: string;
20
+ password: string;
21
+ };
22
+ headers?: Record<string, string>;
23
+ cookies?: Array<{
24
+ name: string;
25
+ value: string;
26
+ }>;
27
+ }
28
+ /** How a project's baseline advances between runs. Absent means `approved`. */
29
+ export type BaselinePolicy = 'approved' | 'rolling';
30
+ /** Frequency at which a project runs unattended. */
31
+ export type ProjectSchedule = 'hourly' | 'daily' | 'weekly';
32
+ /**
33
+ * A saved project as the API returns it.
34
+ *
35
+ * This is a read shape. To change a project, pass {@link ProjectConfigUpdate}
36
+ * to updateProject() — the two differ, because gate credentials are written as
37
+ * an {@link EnvGate} but read back only as a presence flag.
38
+ */
12
39
  export interface ProjectConfig {
13
- orgId?: string;
14
40
  name: string;
15
- testOrigin?: string;
41
+ testOrigin: string;
16
42
  baseOrigin?: string;
17
43
  sitemapUrl?: string;
18
44
  paths?: ProjectPath[];
19
45
  scans?: ProjectScan[];
20
- devices?: string[];
46
+ devices: string[];
21
47
  masks?: string[];
48
+ /** CSS injected before screenshotting, to hide dynamic widgets. Max 4096 characters. */
49
+ customCss?: string;
50
+ /** Presence flag only — the stored credential is never returned. */
51
+ testAuth?: {
52
+ configured: true;
53
+ };
54
+ /** Presence flag only — the stored credential is never returned. */
55
+ baseAuth?: {
56
+ configured: true;
57
+ };
22
58
  concurrency?: number;
23
- createdAt?: string;
24
- updatedAt?: string;
59
+ /** Extra instructions handed to the model that writes change summaries. */
60
+ aiPromptInstructions?: string;
61
+ /** Diff percentage below which a regression is skipped by the AI pass. Defaults to 0.01. */
62
+ aiSummaryThreshold?: number;
63
+ /**
64
+ * `approved` (default): a person accepts a run and its captures become the baseline.
65
+ * `rolling`: every completed managed run advances it, and those runs auto-approve.
66
+ */
67
+ baselinePolicy?: BaselinePolicy;
68
+ /** Set when the project runs unattended. Absent means it only runs when triggered. */
69
+ schedule?: ProjectSchedule;
70
+ /**
71
+ * The UTC hour, 0–23, at which a `daily` or `weekly` project runs — `3` is 03:00 UTC.
72
+ * Absent means the slot anchors to whenever the first sweep picked the project up.
73
+ */
74
+ scheduleHourUtc?: number;
75
+ /**
76
+ * The API key recorded when the schedule was set; every scheduled run is attributed
77
+ * to it. Absent if the schedule was set without an API key, in which case AI
78
+ * summaries are skipped for those runs.
79
+ */
80
+ scheduleKeyId?: string;
81
+ /** When the scheduler last started a run. The next run is due one interval after this. */
82
+ lastScheduledRunAt?: string;
83
+ createdAt: string;
84
+ updatedAt: string;
25
85
  lastRunAt?: string;
26
86
  lastJobId?: string;
87
+ baselineVersion?: string;
27
88
  baselineInvalidatedAt?: string;
28
89
  }
90
+ /**
91
+ * The fields updateProject() accepts.
92
+ *
93
+ * Changing anything that decides what a capture looks like — testOrigin,
94
+ * baseOrigin, sitemapUrl, paths, scans, devices, masks, customCss, or a gate
95
+ * credential — invalidates the stored baselines, and the next run re-captures
96
+ * them instead of comparing. Everything else leaves baselines alone.
97
+ * Re-sending an identical value is not a change.
98
+ */
99
+ export interface ProjectConfigUpdate {
100
+ testOrigin?: string;
101
+ baseOrigin?: string;
102
+ sitemapUrl?: string;
103
+ paths?: ProjectPath[];
104
+ scans?: ProjectScan[];
105
+ devices?: string[];
106
+ masks?: string[];
107
+ /** Max 4096 characters. */
108
+ customCss?: string;
109
+ /** Credentials for the test origin's environment gate. Pass null to clear. */
110
+ testAuth?: EnvGate | null;
111
+ /** Credentials for the base origin's environment gate. Pass null to clear. */
112
+ baseAuth?: EnvGate | null;
113
+ concurrency?: number;
114
+ aiPromptInstructions?: string;
115
+ aiSummaryThreshold?: number;
116
+ baselinePolicy?: BaselinePolicy;
117
+ /**
118
+ * Run this project unattended at this frequency. Requires `baselinePolicy: 'rolling'`
119
+ * on a managed project — the API rejects the combination without it, because on the
120
+ * approved policy an unapproved change is re-reported on every subsequent run.
121
+ * Live-vs-live projects store no baseline and are exempt. Pass null to remove.
122
+ *
123
+ * Without `scheduleHourUtc` the first run starts at the next hourly sweep and the
124
+ * cadence anchors to it.
125
+ */
126
+ schedule?: ProjectSchedule | null;
127
+ /**
128
+ * Pin a `daily` or `weekly` schedule to this UTC hour, 0–23 — `3` means 03:00 UTC.
129
+ * The first run waits for the hour too, rather than firing immediately and settling
130
+ * into the hour from the following day. Hours only, because the scheduler sweeps once
131
+ * an hour; UTC only, because RegressionBot has no timezone to convert from.
132
+ *
133
+ * Rejected on an `hourly` schedule, which already runs every hour, and rejected if the
134
+ * project has no schedule to apply it to — either one already set, or one sent in the
135
+ * same update. Clearing the schedule with
136
+ * `schedule: null` clears this too, so re-enabling later cannot resurrect an hour
137
+ * nobody asked for on that run. Pass null to clear it on its own.
138
+ *
139
+ * A missed slot waits for the next one rather than catching up.
140
+ */
141
+ scheduleHourUtc?: number | null;
142
+ }
29
143
  export interface VRConfig {
30
144
  apiKey?: string;
31
145
  apiUrl?: string;
@@ -52,57 +166,105 @@ export declare const Viewports: {
52
166
  readonly height: 667;
53
167
  };
54
168
  };
169
+ export type VerdictDecision = 'intentional' | 'bug' | 'noise' | 'needs_review';
170
+ /** How one labelled region of a diff was judged against the run's stated intent. */
171
+ export interface RegionVerdict {
172
+ decision: VerdictDecision;
173
+ /** 0–1. */
174
+ confidence: number;
175
+ reasoning: string;
176
+ }
177
+ /** The rolled-up verdict for one result, plus the per-region verdicts behind it. */
178
+ export interface ResultVerdict {
179
+ decision: VerdictDecision;
180
+ minConfidence: number;
181
+ avgConfidence: number;
182
+ /** Keyed by region label (A, B, C…). Empty when the verdict came from the text-only pass. */
183
+ regions: Record<string, RegionVerdict>;
184
+ }
55
185
  export interface RegressionbotSummaryItem {
186
+ /** Region letter (A, B, C…), or an empty string for a whole-page change. */
56
187
  label: string;
188
+ /** Single-sentence description of what changed in that region. */
57
189
  text: string;
190
+ verdict?: RegionVerdict;
58
191
  }
59
192
  /**
60
193
  * Result for a single URL + device comparison.
61
194
  *
62
- * Returned by both getStatus() and getSummary(). Pre-signed image URLs
63
- * (baselineUrl, currentUrl, diffUrl) are only populated by getSummary() —
64
- * they are absent when reading results from getStatus().
195
+ * Returned by getStatus() and, split across `regressions`/`matches`, by
196
+ * getSummary(). Both endpoints pre-sign the image URLs; a URL is null when
197
+ * that image does not exist for this result (no baseline yet, no diff, …).
65
198
  */
66
199
  export interface PageResult {
67
200
  url: string;
68
- /** Device variant tested (e.g. "desktop_chrome", "iphone_12"). */
201
+ /** Whether the capture and comparison succeeded. */
202
+ status: 'SUCCESS' | 'ERROR';
203
+ /** Device variant tested (e.g. "Desktop Chrome", "iPhone 12"). */
69
204
  variantName: string;
70
205
  /** Percentage of pixels that differed from the baseline (0 = identical). */
71
206
  diffPercentage: number;
72
- /** Raw count of differing pixels. */
73
- diffCount: number;
74
207
  /** Perceptual similarity 0-100 (SSIM). */
75
208
  visualMatchScore: number;
209
+ /** True when this capture became a baseline because none existed to compare against. */
210
+ isNewBaseline: boolean;
211
+ /** Why the capture or comparison failed. Present when status is ERROR. */
212
+ errorMessage?: string;
76
213
  /** Plain-English description of what visually changed, generated by RegressionBot. */
77
214
  regressionbotSummary?: RegressionbotSummaryItem[];
215
+ /**
216
+ * Whether the change was judged intentional, a bug, or noise. Only produced when
217
+ * the run carried a {@link RunContext}, and only once summaryStatus is COMPLETE.
218
+ */
219
+ verdict?: ResultVerdict;
220
+ /** Selectors of the elements that changed, when the DOM comparison could identify them. */
221
+ elementsChanged?: string[];
222
+ /**
223
+ * Set when the DOM comparison could not run. The result is a plain pixel diff with
224
+ * no semantic change types, so treat its classifications as absent rather than as
225
+ * "nothing structural changed".
226
+ */
227
+ domAssistSkipReason?: string;
78
228
  /** Pre-signed URL for the stored baseline screenshot. */
79
- baselineUrl?: string;
229
+ baselineUrl: string | null;
80
230
  /** Pre-signed URL for the newly captured screenshot. */
81
- currentUrl?: string;
231
+ currentUrl: string | null;
82
232
  /** Pre-signed URL for the side-by-side annotated diff image. */
83
- diffUrl?: string;
233
+ diffUrl: string | null;
234
+ /**
235
+ * Pre-signed URL for the diff mask. getStatus() always sends the key, null when
236
+ * there is no mask; getSummary() omits it entirely in that case.
237
+ */
238
+ maskUrl?: string | null;
84
239
  }
85
240
  export interface JobProgress {
86
241
  total: number;
87
242
  completed: number;
88
243
  percent: string;
89
244
  }
245
+ /** Progress of the asynchronous AI summary pass. */
246
+ export type SummaryStatus = 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'FAILED';
90
247
  export interface JobStatus {
91
248
  jobId: string;
92
249
  status: 'INITIALIZING' | 'PROCESSING' | 'FINISHING' | 'SUMMARIZING' | 'COMPLETED' | 'APPROVED' | 'FAILED';
93
- summaryStatus?: 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'NONE';
94
- error?: string;
95
- progress?: JobProgress;
96
- executionTime?: number;
97
- createdAt?: string;
98
- /** Partial or complete results. Present once workers start completing. Pre-signed image URLs are not included — call getSummary() for those. */
99
- results?: PageResult[];
250
+ /**
251
+ * PENDING means not yet started, COMPLETE that regressionbotSummary is populated
252
+ * (or that nothing needed one). Poll until COMPLETE before reading summaries.
253
+ */
254
+ summaryStatus: SummaryStatus;
255
+ error: string | null;
256
+ progress: JobProgress;
257
+ executionTime: number;
258
+ createdAt: string;
259
+ /** Partial or complete results. Fills in as workers finish. */
260
+ results: PageResult[];
100
261
  }
101
262
  export interface JobSummary {
102
263
  jobId: string;
103
- status: string;
104
- summaryStatus?: 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'NONE';
105
- error?: string | null;
264
+ /** getSummary() throws until the job reaches one of these. */
265
+ status: 'COMPLETED' | 'APPROVED' | 'FAILED';
266
+ summaryStatus: SummaryStatus;
267
+ error: string | null;
106
268
  totalUrls: number;
107
269
  completedCount: number;
108
270
  /** Overall quality score 0–100 across all tested pages. */
@@ -112,9 +274,9 @@ export interface JobSummary {
112
274
  matchCount: number;
113
275
  newBaselineCount: number;
114
276
  errorCount: number;
115
- /** Pages where a visual regression was detected. Includes pre-signed image URLs. */
277
+ /** Pages where a visual regression was detected. */
116
278
  regressions: PageResult[];
117
- /** Pages that matched the baseline. Includes pre-signed image URLs. */
279
+ /** Pages that matched the baseline. */
118
280
  matches: PageResult[];
119
281
  /** Pages captured for the first time — no baseline existed to compare against. */
120
282
  newBaselines: Array<{
@@ -127,6 +289,49 @@ export interface JobSummary {
127
289
  variantName: string;
128
290
  errorMessage: string;
129
291
  }>;
292
+ /** Whole-job roll-up of how the changes line up with the run's stated intent. */
293
+ intentAssessment: IntentAssessment;
294
+ /** The intent this run was given, echoed back. */
295
+ runContext?: RunContext;
296
+ }
297
+ /**
298
+ * What a run is testing. Supplying it lets RegressionBot judge each change as
299
+ * intentional, a bug, or noise, rather than just reporting that pixels moved.
300
+ */
301
+ export interface RunContext {
302
+ /** Short description of what this run is testing. */
303
+ changeDescription?: string;
304
+ gitCommitSha?: string;
305
+ /** Git commit subject line. */
306
+ gitCommitMessage?: string;
307
+ prTitle?: string;
308
+ /** Markdown. Truncated to 2000 characters by the API. */
309
+ prDescription?: string;
310
+ /** Explicit list of expected visual changes. */
311
+ expectedChanges?: string[];
312
+ /** Files or component names changed. */
313
+ scope?: string[];
314
+ }
315
+ export interface IntentAssessment {
316
+ /** False when the run carried no RunContext, in which case the counts are all zero. */
317
+ intentProvided: boolean;
318
+ bugCount: number;
319
+ intentionalCount: number;
320
+ noiseCount: number;
321
+ needsReviewCount: number;
322
+ /** True when every regression is accounted for by the stated intent. */
323
+ allAccountedFor: boolean;
324
+ /** One line summarising the job, safe to print straight into CI output. */
325
+ summary: string;
326
+ }
327
+ export interface ApproveResult {
328
+ message: string;
329
+ jobId: string;
330
+ approvedUrlsCount: number;
331
+ /** Present when some captures could not be promoted. */
332
+ failedCount?: number;
333
+ /** Pages skipped because another job updated their baseline first. */
334
+ conflictedUrls?: string[];
130
335
  }
131
336
  export interface JobAiSummary {
132
337
  message: string;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@regressionbot/sdk",
3
- "version": "1.2.3",
3
+ "version": "2.0.0",
4
4
  "description": "The official SDK for regressionbot.com - the simplest way to automate visual regression testing.",
5
5
  "homepage": "https://regressionbot.com",
6
6
  "repository": {