@regressionbot/sdk 2.2.0 → 2.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +85 -6
- package/dist/cli.d.ts +10 -2
- package/dist/cli.js +42 -4
- package/dist/index.d.ts +23 -4
- package/dist/index.js +62 -9
- package/dist/types.d.ts +51 -4
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -74,6 +74,43 @@ the builder, intent-aware verdicts, environment gates, baseline policies, and
|
|
|
74
74
|
scheduling. `scheduleHourUtc` needs API 2.7.0 or later — on an older API the
|
|
75
75
|
field is accepted and ignored, and the schedule stays anchored to its first run.
|
|
76
76
|
|
|
77
|
+
### 2.3.0
|
|
78
|
+
|
|
79
|
+
Needs API 2.9.0 or later. On an older API the verdict fields are simply absent, with
|
|
80
|
+
one exception called out below.
|
|
81
|
+
|
|
82
|
+
- **`RESOLVED` is a terminal status, and `waitForCompletion` now stops on it.** A run
|
|
83
|
+
reaches `APPROVED` only when every page carrying a diff was approved; decide them all
|
|
84
|
+
and reject one and it lands on `RESOLVED` instead. The old terminal check knew
|
|
85
|
+
`COMPLETED` and `APPROVED` only, so a triaged run with a single rejection polled until
|
|
86
|
+
the caller gave up. API 2.9.0 made that the common way such a run ends.
|
|
87
|
+
- **`approvePage(url, variantName, note?)` and `rejectPage(...)`** decide one page and
|
|
88
|
+
leave the rest of the job alone. The optional `note` records what the verdict missed,
|
|
89
|
+
and is worth sending when you are overruling it. Both refuse a url without a
|
|
90
|
+
`variantName`: the API gates its per-page path on having both, and a half-identified
|
|
91
|
+
page falls through to approving the whole job.
|
|
92
|
+
- **`approve(where)` throws if it reaches an API that does not support the filter.**
|
|
93
|
+
This is the exception. An older API reads `jobId`, ignores the rest of the body, and
|
|
94
|
+
approves EVERY page — including the ones the verdict called bugs — while answering
|
|
95
|
+
like a success. `skippedUrlsCount` is the tell: the filter path always reports it,
|
|
96
|
+
even as 0, and no other path reports it at all. The error says plainly that the
|
|
97
|
+
baselines have already moved, because by then they have.
|
|
98
|
+
- **Verdict and triage fields typed.** On a regression: `basis`, `coveredBy`,
|
|
99
|
+
`reasoning` and a deprecated `judgedAt`. On a page a person decided: `triageStatus`,
|
|
100
|
+
`triagedAt`, `triagedBy`, `triagedVerdict` and `triageNote`. On the job:
|
|
101
|
+
`intentAssessment.decision` and `intentProvided`.
|
|
102
|
+
- **`comparisonMode`** on the job status says whether the run compared two live origins
|
|
103
|
+
or one origin against a stored baseline.
|
|
104
|
+
|
|
105
|
+
### 2.2.1
|
|
106
|
+
|
|
107
|
+
- **A builder run with no `check()` or `scan()` no longer invents a home path.** It
|
|
108
|
+
used to send `paths: [{ path: '/', label: 'Home' }]`, which the API compared to the
|
|
109
|
+
saved project and rejected whenever the project stored anything other than exactly
|
|
110
|
+
`/`. Now `paths` is omitted, and the API fills it from the saved project. Needs an
|
|
111
|
+
API that fills omitted run params from the stored project (shipped alongside this
|
|
112
|
+
release); on an older API a scope-less run captures the home page only.
|
|
113
|
+
|
|
77
114
|
### 2.2.0
|
|
78
115
|
|
|
79
116
|
- **Six metadata `ChangeType`s added** (`meta-edit`, `meta-insert`, `meta-delete`,
|
|
@@ -300,7 +337,6 @@ intent-aware too:
|
|
|
300
337
|
|
|
301
338
|
```typescript
|
|
302
339
|
const job = await rb.runProject('marketing-site-v2', {
|
|
303
|
-
testOrigin: process.env.VERCEL_PREVIEW_URL,
|
|
304
340
|
runContext: { changeDescription: 'Restyle the pricing table' },
|
|
305
341
|
});
|
|
306
342
|
```
|
|
@@ -324,7 +360,11 @@ const job = await rb
|
|
|
324
360
|
|
|
325
361
|
### Progress Tracking
|
|
326
362
|
|
|
327
|
-
|
|
363
|
+
`waitForCompletion` returns on `COMPLETED`, `APPROVED` or `RESOLVED`, and throws on
|
|
364
|
+
`FAILED`. `APPROVED` means every page carrying a diff was approved; `RESOLVED` means
|
|
365
|
+
they were all decided and at least one was rejected.
|
|
366
|
+
|
|
367
|
+
Pass a callback to receive status updates while the job runs:
|
|
328
368
|
|
|
329
369
|
```typescript
|
|
330
370
|
const status = await job.waitForCompletion(3000, (s) => {
|
|
@@ -352,6 +392,34 @@ const result = await job.approve();
|
|
|
352
392
|
console.log(`Approved ${result.approvedUrlsCount} URLs.`);
|
|
353
393
|
```
|
|
354
394
|
|
|
395
|
+
When the run carried an intent (`.withContext()`), approve only the pages the
|
|
396
|
+
verdict cleared and leave the flagged ones for a person:
|
|
397
|
+
|
|
398
|
+
```typescript
|
|
399
|
+
const summary = await job.getSummary();
|
|
400
|
+
if (summary.intentAssessment.decision !== 'pass') {
|
|
401
|
+
const result = await job.approve({ decision: ['intentional', 'noise'] });
|
|
402
|
+
console.log(`Approved ${result.approvedUrlsCount}, ${result.skippedUrlsCount} left for review.`);
|
|
403
|
+
}
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
`intentAssessment.decision` is `pass`, `review`, `fail`, or `not_judged` when the
|
|
407
|
+
run carried no intent. Each regression's `verdict` carries `reasoning`, `basis`
|
|
408
|
+
(`measured` or `described`) and, for `intentional`, the words of the intent it
|
|
409
|
+
`coveredBy`. Both need API 2.9.0 or later.
|
|
410
|
+
|
|
411
|
+
To decide one page at a time, name the page and the device:
|
|
412
|
+
|
|
413
|
+
```typescript
|
|
414
|
+
await job.rejectPage('https://example.com/pricing', 'Desktop Chrome', 'the nav lost a link');
|
|
415
|
+
await job.approvePage('https://example.com/pricing', 'iPhone 12');
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
The third argument is a note, and it is worth sending when you are overruling the
|
|
419
|
+
verdict — approving a page it called a bug, or rejecting one it cleared. What you
|
|
420
|
+
write is stored on that page as `triageNote`, beside `triageStatus`, `triagedAt`,
|
|
421
|
+
`triagedBy` and the `triagedVerdict` you overruled.
|
|
422
|
+
|
|
355
423
|
### Saved Projects
|
|
356
424
|
|
|
357
425
|
Projects let you save a test configuration in the RegressionBot dashboard and trigger runs against it without re-specifying every option.
|
|
@@ -365,10 +433,10 @@ console.log(projects.map(p => p.name));
|
|
|
365
433
|
const project = await rb.getProject('marketing-site-v2');
|
|
366
434
|
console.log(project);
|
|
367
435
|
|
|
368
|
-
// Trigger a run from a saved project
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
436
|
+
// Trigger a run from a saved project. Any capture field you pass is checked
|
|
437
|
+
// against the saved config and rejected if it differs; leave it out to use
|
|
438
|
+
// the saved value. Change the config with updateProject().
|
|
439
|
+
const job = await rb.runProject('marketing-site-v2');
|
|
372
440
|
|
|
373
441
|
const status = await job.waitForCompletion();
|
|
374
442
|
const summary = await job.getSummary();
|
|
@@ -525,6 +593,17 @@ Promote the current screenshots of a job to be the new baselines.
|
|
|
525
593
|
npx @regressionbot/sdk approve <jobId>
|
|
526
594
|
```
|
|
527
595
|
|
|
596
|
+
When the run carried an intent, `--decision` approves only the pages whose verdict is in
|
|
597
|
+
the list and leaves the rest pending:
|
|
598
|
+
```bash
|
|
599
|
+
npx @regressionbot/sdk approve <jobId> --decision intentional,noise
|
|
600
|
+
```
|
|
601
|
+
|
|
602
|
+
It prints how many changed pages were left for a person. Treat a number above zero as a
|
|
603
|
+
build that is not done: those are the pages the verdict did not clear, plus any it never
|
|
604
|
+
assessed. An unknown value stops the command rather than being sent, because an API that
|
|
605
|
+
does not understand the filter approves the whole job instead.
|
|
606
|
+
|
|
528
607
|
## Examples & Integrations
|
|
529
608
|
|
|
530
609
|
Check out the [examples/](./examples/) directory for real-world integration guides:
|
package/dist/cli.d.ts
CHANGED
|
@@ -1,8 +1,16 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
import { PageResult, RunContext } from './types';
|
|
2
|
+
import { PageResult, RunContext, VerdictDecision } from './types';
|
|
3
3
|
declare function printRegression(r: PageResult): void;
|
|
4
4
|
type FailOn = 'any' | 'unintended';
|
|
5
5
|
declare function parseFailOn(raw: unknown): FailOn;
|
|
6
|
+
/**
|
|
7
|
+
* `--decision intentional,noise` -> the approve filter, or undefined for no filter.
|
|
8
|
+
*
|
|
9
|
+
* Validated here rather than at the API, because the failure is asymmetric: a typo the API
|
|
10
|
+
* rejects costs a round trip, but a typo that reached a filter-blind API would approve the
|
|
11
|
+
* whole job. Unknown values stop the command.
|
|
12
|
+
*/
|
|
13
|
+
declare function parseDecisions(raw: unknown): VerdictDecision[] | undefined;
|
|
6
14
|
/**
|
|
7
15
|
* Whether one regression should fail the build under `--fail-on unintended`.
|
|
8
16
|
*
|
|
@@ -25,4 +33,4 @@ declare function selectBlocking(regressions: PageResult[], failOn: FailOn): {
|
|
|
25
33
|
};
|
|
26
34
|
declare function buildRunContext(options: any): RunContext | undefined;
|
|
27
35
|
declare function parseArgs(args: string[]): any;
|
|
28
|
-
export { parseArgs, parseFailOn, isBlocking, selectBlocking, buildRunContext, printRegression };
|
|
36
|
+
export { parseArgs, parseFailOn, parseDecisions, isBlocking, selectBlocking, buildRunContext, printRegression };
|
package/dist/cli.js
CHANGED
|
@@ -36,6 +36,7 @@ var __importStar = (this && this.__importStar) || (function () {
|
|
|
36
36
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
37
37
|
exports.parseArgs = parseArgs;
|
|
38
38
|
exports.parseFailOn = parseFailOn;
|
|
39
|
+
exports.parseDecisions = parseDecisions;
|
|
39
40
|
exports.isBlocking = isBlocking;
|
|
40
41
|
exports.selectBlocking = selectBlocking;
|
|
41
42
|
exports.buildRunContext = buildRunContext;
|
|
@@ -83,6 +84,27 @@ function parseFailOn(raw) {
|
|
|
83
84
|
return raw;
|
|
84
85
|
throw new UsageError("--fail-on takes 'any' or 'unintended'.");
|
|
85
86
|
}
|
|
87
|
+
const DECISIONS = ['intentional', 'bug', 'noise', 'needs_review'];
|
|
88
|
+
/**
|
|
89
|
+
* `--decision intentional,noise` -> the approve filter, or undefined for no filter.
|
|
90
|
+
*
|
|
91
|
+
* Validated here rather than at the API, because the failure is asymmetric: a typo the API
|
|
92
|
+
* rejects costs a round trip, but a typo that reached a filter-blind API would approve the
|
|
93
|
+
* whole job. Unknown values stop the command.
|
|
94
|
+
*/
|
|
95
|
+
function parseDecisions(raw) {
|
|
96
|
+
if (raw === undefined)
|
|
97
|
+
return undefined;
|
|
98
|
+
if (typeof raw !== 'string' || !raw.trim()) {
|
|
99
|
+
throw new UsageError(`--decision takes a comma-separated list of: ${DECISIONS.join(', ')}.`);
|
|
100
|
+
}
|
|
101
|
+
const wanted = raw.split(',').map(d => d.trim()).filter(Boolean);
|
|
102
|
+
const bad = wanted.filter(d => !DECISIONS.includes(d));
|
|
103
|
+
if (bad.length > 0) {
|
|
104
|
+
throw new UsageError(`--decision does not take ${bad.join(', ')}. Use: ${DECISIONS.join(', ')}.`);
|
|
105
|
+
}
|
|
106
|
+
return wanted;
|
|
107
|
+
}
|
|
86
108
|
/**
|
|
87
109
|
* Whether one regression should fail the build under `--fail-on unintended`.
|
|
88
110
|
*
|
|
@@ -185,7 +207,7 @@ async function main() {
|
|
|
185
207
|
else if (command === 'approve') {
|
|
186
208
|
if (!param)
|
|
187
209
|
throw new Error('Job ID is required for approve command.');
|
|
188
|
-
await approveJob(param);
|
|
210
|
+
await approveJob(param, parseDecisions(argv['decision']));
|
|
189
211
|
}
|
|
190
212
|
else if (command.startsWith('http')) {
|
|
191
213
|
// Implicit test command
|
|
@@ -214,6 +236,8 @@ Usage:
|
|
|
214
236
|
Use --download to save the diff image locally.
|
|
215
237
|
Use --download-full to save baseline and current images too.
|
|
216
238
|
npx regressionbot approve <jobId> Approve a job's results as new baselines.
|
|
239
|
+
Use --decision to approve only what the verdict
|
|
240
|
+
cleared, e.g. --decision intentional,noise.
|
|
217
241
|
|
|
218
242
|
Options for <url>:
|
|
219
243
|
--project <id> Required project ID.
|
|
@@ -406,11 +430,25 @@ Errors: ${summary.errorCount}
|
|
|
406
430
|
});
|
|
407
431
|
}
|
|
408
432
|
}
|
|
409
|
-
async function approveJob(jobId) {
|
|
410
|
-
console.log(`Approving baselines for job: ${jobId}...`);
|
|
433
|
+
async function approveJob(jobId, decision) {
|
|
411
434
|
const job = sdk.job(jobId);
|
|
412
|
-
|
|
435
|
+
if (!decision) {
|
|
436
|
+
console.log(`Approving every page in job: ${jobId}...`);
|
|
437
|
+
const res = await job.approve();
|
|
438
|
+
console.log(`Success! ${res.message}`);
|
|
439
|
+
return;
|
|
440
|
+
}
|
|
441
|
+
console.log(`Approving pages the verdict called ${decision.join(' or ')} in job: ${jobId}...`);
|
|
442
|
+
const res = await job.approve({ decision });
|
|
413
443
|
console.log(`Success! ${res.message}`);
|
|
444
|
+
// Left pending on purpose, and the reason a CI step should not go green here: these are
|
|
445
|
+
// the changed pages the verdict did not clear, plus any it never assessed.
|
|
446
|
+
if (res.skippedUrlsCount) {
|
|
447
|
+
console.log(`${res.skippedUrlsCount} changed page(s) left for a person to decide.`);
|
|
448
|
+
}
|
|
449
|
+
if (res.conflictedUrls?.length) {
|
|
450
|
+
console.log(`${res.conflictedUrls.length} page(s) skipped: another job moved their baseline first.`);
|
|
451
|
+
}
|
|
414
452
|
}
|
|
415
453
|
// Only run when invoked as a command, so the pure helpers below can be imported and
|
|
416
454
|
// tested without the CLI executing on require.
|
package/dist/index.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult, Change, ChangeType, StyleDelta, ChangeBox } from './types';
|
|
1
|
+
import { VRConfig, Viewport, Viewports, JobStatus, JobSummary, JobProgress, PageResult, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult, ApproveWhere, Change, ChangeType, StyleDelta, ChangeBox } from './types';
|
|
2
2
|
import { sanitizeFilename, sanitizeUrlToPath } from './security';
|
|
3
3
|
export { sanitizeFilename, sanitizeUrlToPath, Viewports };
|
|
4
|
-
export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult, Change, ChangeType, StyleDelta, ChangeBox };
|
|
4
|
+
export type { PageResult, JobProgress, JobStatus, JobSummary, Viewport, VRConfig, ProjectConfig, ProjectPath, ProjectScan, JobAiSummary, RegionVerdict, ResultVerdict, VerdictDecision, RegressionbotSummaryItem, RunContext, IntentAssessment, BaselinePolicy, ProjectSchedule, ProjectConfigUpdate, EnvGate, SummaryStatus, ApproveResult, ApproveWhere, Change, ChangeType, StyleDelta, ChangeBox };
|
|
5
5
|
export declare class RegressionBot {
|
|
6
6
|
private apiKey;
|
|
7
7
|
private apiUrl;
|
|
@@ -24,6 +24,10 @@ export declare class RegressionBot {
|
|
|
24
24
|
listProjects(): Promise<ProjectConfig[]>;
|
|
25
25
|
/**
|
|
26
26
|
* Trigger a new job run using a saved project configuration.
|
|
27
|
+
*
|
|
28
|
+
* Capture fields passed here are not overrides: the API compares each one to the
|
|
29
|
+
* saved config and rejects the run if it differs. Omit a field to use the saved
|
|
30
|
+
* value; change the saved value with updateProject().
|
|
27
31
|
*/
|
|
28
32
|
runProject(projectName: string, options?: {
|
|
29
33
|
testOrigin?: string;
|
|
@@ -108,7 +112,22 @@ export declare class JobHandle {
|
|
|
108
112
|
constructor(sdk: RegressionBot, jobId: string);
|
|
109
113
|
getStatus(): Promise<JobStatus>;
|
|
110
114
|
getSummary(): Promise<JobSummary>;
|
|
111
|
-
|
|
115
|
+
/**
|
|
116
|
+
* Promote this job's captures to baselines. Pass `where` to approve only the pages whose
|
|
117
|
+
* verdict is in the list — `{ decision: ['intentional', 'noise'] }` approves what the verdict
|
|
118
|
+
* cleared and leaves the flagged pages for a person.
|
|
119
|
+
*/
|
|
120
|
+
approve(where?: ApproveWhere): Promise<ApproveResult>;
|
|
121
|
+
/**
|
|
122
|
+
* Approve one page's capture, leaving the rest of the job alone.
|
|
123
|
+
*
|
|
124
|
+
* `note` records what the verdict missed, and is worth sending when you are overruling it —
|
|
125
|
+
* approving a page it called a bug, or rejecting one it cleared.
|
|
126
|
+
*/
|
|
127
|
+
approvePage(url: string, variantName: string, note?: string): Promise<ApproveResult>;
|
|
128
|
+
/** Reject one page, leaving its baseline where it is. See `approvePage` for `note`. */
|
|
129
|
+
rejectPage(url: string, variantName: string, note?: string): Promise<ApproveResult>;
|
|
130
|
+
private decidePage;
|
|
112
131
|
/**
|
|
113
132
|
* Generate AI summaries on-demand for all regressions in a completed job.
|
|
114
133
|
*/
|
|
@@ -122,7 +141,7 @@ export declare class JobHandle {
|
|
|
122
141
|
baseDir?: string;
|
|
123
142
|
}): Promise<void>;
|
|
124
143
|
/**
|
|
125
|
-
* Poll until the job reaches a terminal state (COMPLETED, APPROVED, or FAILED).
|
|
144
|
+
* Poll until the job reaches a terminal state (COMPLETED, APPROVED, RESOLVED or FAILED).
|
|
126
145
|
* @param intervalMs Polling interval in milliseconds. Defaults to 2000.
|
|
127
146
|
* @param callback Optional callback invoked on each status poll.
|
|
128
147
|
* @param options.waitForSummaries If true, keeps polling until AI summaries are fully populated before returning.
|
package/dist/index.js
CHANGED
|
@@ -48,6 +48,10 @@ class RegressionBot {
|
|
|
48
48
|
}
|
|
49
49
|
/**
|
|
50
50
|
* Trigger a new job run using a saved project configuration.
|
|
51
|
+
*
|
|
52
|
+
* Capture fields passed here are not overrides: the API compares each one to the
|
|
53
|
+
* saved config and rejects the run if it differs. Omit a field to use the saved
|
|
54
|
+
* value; change the saved value with updateProject().
|
|
51
55
|
*/
|
|
52
56
|
async runProject(projectName, options = {}) {
|
|
53
57
|
const res = await this._request(`/project/${encodeURIComponent(projectName)}/run`, 'POST', options);
|
|
@@ -173,10 +177,6 @@ class JobBuilder {
|
|
|
173
177
|
if (!this.manifest.projectId && !this.manifest.baseOrigin) {
|
|
174
178
|
throw new Error('Project ID is required. Use .forProject("id") or provide an origin to compare against using .against()');
|
|
175
179
|
}
|
|
176
|
-
// If no checks/scans provided, default to root
|
|
177
|
-
if (this.manifest.checks.length === 0 && this.manifest.scans.length === 0) {
|
|
178
|
-
this.manifest.checks.push({ path: '/', label: 'Home' });
|
|
179
|
-
}
|
|
180
180
|
// Every field present here is compared against the saved project config, and a
|
|
181
181
|
// value that differs fails the run rather than overriding it. So a field the
|
|
182
182
|
// caller never set must stay absent: an empty `devices` or a `concurrency` this
|
|
@@ -187,7 +187,7 @@ class JobBuilder {
|
|
|
187
187
|
sitemapUrl: this.manifest.sitemapUrl,
|
|
188
188
|
baseOrigin: this.manifest.baseOrigin,
|
|
189
189
|
devices: this.manifest.variants.length > 0 ? this.manifest.variants : undefined,
|
|
190
|
-
paths: this.manifest.checks,
|
|
190
|
+
paths: this.manifest.checks.length > 0 ? this.manifest.checks : undefined,
|
|
191
191
|
scans: this.manifest.scans.length > 0 ? this.manifest.scans : undefined,
|
|
192
192
|
concurrency: this.manifest.concurrency,
|
|
193
193
|
autoApprove: this.manifest.autoApprove,
|
|
@@ -213,8 +213,58 @@ class JobHandle {
|
|
|
213
213
|
async getSummary() {
|
|
214
214
|
return this.sdk._request(`/job/${encodeURIComponent(this.jobId)}/summary`);
|
|
215
215
|
}
|
|
216
|
-
|
|
217
|
-
|
|
216
|
+
/**
|
|
217
|
+
* Promote this job's captures to baselines. Pass `where` to approve only the pages whose
|
|
218
|
+
* verdict is in the list — `{ decision: ['intentional', 'noise'] }` approves what the verdict
|
|
219
|
+
* cleared and leaves the flagged pages for a person.
|
|
220
|
+
*/
|
|
221
|
+
async approve(where) {
|
|
222
|
+
const result = await this.sdk._request('/approve', 'POST', {
|
|
223
|
+
jobId: this.jobId,
|
|
224
|
+
...(where ? { where } : {}),
|
|
225
|
+
});
|
|
226
|
+
// An API that predates the filter reads `jobId` and ignores everything else, so a body
|
|
227
|
+
// it does not understand approves EVERY page — including the ones the verdict called
|
|
228
|
+
// bugs — and answers like a success. The tell is `skippedUrlsCount`: the filter path
|
|
229
|
+
// always reports it, even as 0, and no other path reports it at all.
|
|
230
|
+
//
|
|
231
|
+
// Thrown after the fact because that is when it can be known, and the message has to
|
|
232
|
+
// say so: the baselines have already moved, and someone has to look.
|
|
233
|
+
if (where && result.skippedUrlsCount === undefined) {
|
|
234
|
+
throw new Error('approve(where) reached an API that does not support the filter, so EVERY page in '
|
|
235
|
+
+ `job ${this.jobId} was approved and its baselines have already moved. Check the `
|
|
236
|
+
+ 'job and re-reject anything that should not have been promoted. Upgrade the API '
|
|
237
|
+
+ 'to 2.9.0 or later, or decide the pages one at a time with approvePage and '
|
|
238
|
+
+ 'rejectPage.');
|
|
239
|
+
}
|
|
240
|
+
return result;
|
|
241
|
+
}
|
|
242
|
+
/**
|
|
243
|
+
* Approve one page's capture, leaving the rest of the job alone.
|
|
244
|
+
*
|
|
245
|
+
* `note` records what the verdict missed, and is worth sending when you are overruling it —
|
|
246
|
+
* approving a page it called a bug, or rejecting one it cleared.
|
|
247
|
+
*/
|
|
248
|
+
async approvePage(url, variantName, note) {
|
|
249
|
+
return this.decidePage('approve', url, variantName, note);
|
|
250
|
+
}
|
|
251
|
+
/** Reject one page, leaving its baseline where it is. See `approvePage` for `note`. */
|
|
252
|
+
async rejectPage(url, variantName, note) {
|
|
253
|
+
return this.decidePage('reject', url, variantName, note);
|
|
254
|
+
}
|
|
255
|
+
async decidePage(action, url, variantName, note) {
|
|
256
|
+
// Both halves are required together: `url` without `variantName` matches nothing, and the
|
|
257
|
+
// API answers that with a whole-job approval rather than an error.
|
|
258
|
+
if (!url || !variantName) {
|
|
259
|
+
throw new Error('approvePage and rejectPage need both a url and a variantName.');
|
|
260
|
+
}
|
|
261
|
+
return this.sdk._request('/approve', 'POST', {
|
|
262
|
+
jobId: this.jobId,
|
|
263
|
+
url,
|
|
264
|
+
variantName,
|
|
265
|
+
action,
|
|
266
|
+
...(note ? { note } : {}),
|
|
267
|
+
});
|
|
218
268
|
}
|
|
219
269
|
/**
|
|
220
270
|
* Generate AI summaries on-demand for all regressions in a completed job.
|
|
@@ -289,7 +339,7 @@ class JobHandle {
|
|
|
289
339
|
}
|
|
290
340
|
}
|
|
291
341
|
/**
|
|
292
|
-
* Poll until the job reaches a terminal state (COMPLETED, APPROVED, or FAILED).
|
|
342
|
+
* Poll until the job reaches a terminal state (COMPLETED, APPROVED, RESOLVED or FAILED).
|
|
293
343
|
* @param intervalMs Polling interval in milliseconds. Defaults to 2000.
|
|
294
344
|
* @param callback Optional callback invoked on each status poll.
|
|
295
345
|
* @param options.waitForSummaries If true, keeps polling until AI summaries are fully populated before returning.
|
|
@@ -299,7 +349,10 @@ class JobHandle {
|
|
|
299
349
|
const status = await this.getStatus();
|
|
300
350
|
if (callback)
|
|
301
351
|
callback(status);
|
|
302
|
-
|
|
352
|
+
// RESOLVED is terminal too: every page carrying a diff was decided and at least one
|
|
353
|
+
// was rejected. Omitting it polled until the caller's timeout on any run with a
|
|
354
|
+
// rejection, which this release made the common way a triaged run ends.
|
|
355
|
+
if (status.status === 'COMPLETED' || status.status === 'APPROVED' || status.status === 'RESOLVED') {
|
|
303
356
|
if (options?.waitForSummaries) {
|
|
304
357
|
if (status.summaryStatus !== 'PENDING' && status.summaryStatus !== 'PROCESSING') {
|
|
305
358
|
return status;
|
package/dist/types.d.ts
CHANGED
|
@@ -180,12 +180,28 @@ export interface RegionVerdict {
|
|
|
180
180
|
confidence: number;
|
|
181
181
|
reasoning: string;
|
|
182
182
|
}
|
|
183
|
-
/**
|
|
183
|
+
/**
|
|
184
|
+
* How one page's change measures against the intent the run was given. Present only when the
|
|
185
|
+
* job carried a {@link RunContext}. A verdict is a recommendation: {@link Job.approve} and
|
|
186
|
+
* rejecting remain the only actions that move a baseline.
|
|
187
|
+
*/
|
|
184
188
|
export interface ResultVerdict {
|
|
185
189
|
decision: VerdictDecision;
|
|
190
|
+
/** The confidence in the decision, 0–1. */
|
|
186
191
|
minConfidence: number;
|
|
192
|
+
/** One sentence saying why. */
|
|
193
|
+
reasoning?: string;
|
|
194
|
+
/**
|
|
195
|
+
* For `intentional` only: the words of the intent, quoted verbatim, that name this change.
|
|
196
|
+
* An intentional verdict whose quoted words are not in the intent is downgraded to
|
|
197
|
+
* `needs_review` before it is stored.
|
|
198
|
+
*/
|
|
199
|
+
coveredBy?: string;
|
|
200
|
+
/** When the decision was made. */
|
|
201
|
+
judgedAt?: string;
|
|
202
|
+
/** @deprecated Equal to `minConfidence`. Kept for older clients. */
|
|
187
203
|
avgConfidence: number;
|
|
188
|
-
/**
|
|
204
|
+
/** @deprecated Always empty. Kept for older clients. */
|
|
189
205
|
regions: Record<string, RegionVerdict>;
|
|
190
206
|
/**
|
|
191
207
|
* What the judgement was made from.
|
|
@@ -337,6 +353,14 @@ export interface PageResult {
|
|
|
337
353
|
* the run carried a {@link RunContext}, and only once summaryStatus is COMPLETE.
|
|
338
354
|
*/
|
|
339
355
|
verdict?: ResultVerdict;
|
|
356
|
+
/** A person's Approve or Reject on this page, written by {@link Job.approve} or the dashboard. */
|
|
357
|
+
triageStatus?: 'PENDING' | 'APPROVING' | 'APPROVED' | 'REJECTED';
|
|
358
|
+
triagedAt?: string;
|
|
359
|
+
triagedBy?: string;
|
|
360
|
+
/** The verdict's decision at the moment of the click; null when the page had none. */
|
|
361
|
+
triagedVerdict?: VerdictDecision | null;
|
|
362
|
+
/** Optional, from the person, when their action disagreed with the verdict. */
|
|
363
|
+
triageNote?: string;
|
|
340
364
|
/**
|
|
341
365
|
* The exact edits the engine computed from the two documents — element, text before,
|
|
342
366
|
* text after, changed styles. Absent when the DOM comparison could not run on the page,
|
|
@@ -381,7 +405,12 @@ export interface JobProgress {
|
|
|
381
405
|
export type SummaryStatus = 'PENDING' | 'PROCESSING' | 'COMPLETE' | 'FAILED';
|
|
382
406
|
export interface JobStatus {
|
|
383
407
|
jobId: string;
|
|
384
|
-
|
|
408
|
+
/**
|
|
409
|
+
* Terminal states are COMPLETED, APPROVED, RESOLVED and FAILED. APPROVED means every page
|
|
410
|
+
* carrying a diff was approved; RESOLVED means they were all decided and at least one was
|
|
411
|
+
* rejected. RESOLVED was missing here, and waitForCompletion polled past it forever.
|
|
412
|
+
*/
|
|
413
|
+
status: 'INITIALIZING' | 'PROCESSING' | 'FINISHING' | 'SUMMARIZING' | 'COMPLETED' | 'APPROVED' | 'RESOLVED' | 'FAILED';
|
|
385
414
|
/**
|
|
386
415
|
* PENDING means not yet started, COMPLETE that regressionbotSummary is populated
|
|
387
416
|
* (or that nothing needed one). Poll until COMPLETE before reading summaries.
|
|
@@ -391,13 +420,17 @@ export interface JobStatus {
|
|
|
391
420
|
progress: JobProgress;
|
|
392
421
|
executionTime: number;
|
|
393
422
|
createdAt: string;
|
|
423
|
+
/** Whether the run carried an intent. Without one no page gets a verdict. */
|
|
424
|
+
intentProvided?: boolean;
|
|
425
|
+
/** Whether this run compared two live origins or one origin against a stored baseline. */
|
|
426
|
+
comparisonMode?: 'live-vs-live' | 'managed';
|
|
394
427
|
/** Partial or complete results. Fills in as workers finish. */
|
|
395
428
|
results: PageResult[];
|
|
396
429
|
}
|
|
397
430
|
export interface JobSummary {
|
|
398
431
|
jobId: string;
|
|
399
432
|
/** getSummary() throws until the job reaches one of these. */
|
|
400
|
-
status: 'COMPLETED' | 'APPROVED' | 'FAILED';
|
|
433
|
+
status: 'COMPLETED' | 'APPROVED' | 'RESOLVED' | 'FAILED';
|
|
401
434
|
summaryStatus: SummaryStatus;
|
|
402
435
|
error: string | null;
|
|
403
436
|
totalUrls: number;
|
|
@@ -448,6 +481,12 @@ export interface RunContext {
|
|
|
448
481
|
scope?: string[];
|
|
449
482
|
}
|
|
450
483
|
export interface IntentAssessment {
|
|
484
|
+
/**
|
|
485
|
+
* The one field a CI step reads. `fail`: a page contradicts the intent. `review`: a page
|
|
486
|
+
* needs a person, or was never assessed. `pass`: every changed page is accounted for.
|
|
487
|
+
* `not_judged`: the run carried no intent, so nothing was assessed.
|
|
488
|
+
*/
|
|
489
|
+
decision: 'pass' | 'review' | 'fail' | 'not_judged';
|
|
451
490
|
/** False when the run carried no RunContext, in which case the counts are all zero. */
|
|
452
491
|
intentProvided: boolean;
|
|
453
492
|
bugCount: number;
|
|
@@ -459,14 +498,22 @@ export interface IntentAssessment {
|
|
|
459
498
|
/** One line summarising the job, safe to print straight into CI output. */
|
|
460
499
|
summary: string;
|
|
461
500
|
}
|
|
501
|
+
/** Approve only the pages whose verdict is in the list. A job-level filter. */
|
|
502
|
+
export interface ApproveWhere {
|
|
503
|
+
decision: VerdictDecision[];
|
|
504
|
+
}
|
|
462
505
|
export interface ApproveResult {
|
|
463
506
|
message: string;
|
|
464
507
|
jobId: string;
|
|
465
508
|
approvedUrlsCount: number;
|
|
509
|
+
/** With `where` only: changed pages left pending because their verdict was outside the list or absent. */
|
|
510
|
+
skippedUrlsCount?: number;
|
|
466
511
|
/** Present when some captures could not be promoted. */
|
|
467
512
|
failedCount?: number;
|
|
468
513
|
/** Pages skipped because another job updated their baseline first. */
|
|
469
514
|
conflictedUrls?: string[];
|
|
515
|
+
/** One page only: what approvePage or rejectPage wrote on that page's result. */
|
|
516
|
+
triageStatus?: 'APPROVED' | 'REJECTED';
|
|
470
517
|
}
|
|
471
518
|
export interface JobAiSummary {
|
|
472
519
|
message: string;
|
package/package.json
CHANGED