@cappa/cli 0.13.3 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.cjs +445 -22
- package/dist/index.cjs.map +1 -1
- package/dist/index.mjs +449 -21
- package/dist/index.mjs.map +1 -1
- package/integrations/opencode/agents/cappa-judge.md +61 -0
- package/integrations/opencode/commands/cappa-approve.md +19 -0
- package/integrations/opencode/commands/cappa-judge.md +14 -0
- package/integrations/opencode/commands/cappa-review.md +31 -0
- package/integrations/skills/cappa/JUDGING.md +55 -0
- package/integrations/skills/cappa/MASKING.md +40 -0
- package/integrations/skills/cappa/SKILL.md +95 -0
- package/package.json +7 -5
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# cappa — judging changes
|
|
2
|
+
|
|
3
|
+
Used when `cappa agent check --json` reports `pendingJudgments > 0`: the heuristic labelled those
|
|
4
|
+
changes `ambiguous` and is deferring to you.
|
|
5
|
+
|
|
6
|
+
## Look for one shared cause first
|
|
7
|
+
Before opening anything, group the pending `results[]` by `verdict.headline`. Headlines are built so
|
|
8
|
+
that the same kind of change produces the same text — `1 content-change @ bottom-right (0.02%, low)`
|
|
9
|
+
across many screenshots is almost certainly one shared element (footer date, version badge,
|
|
10
|
+
theme toggle, a shared component everyone renders). Open one or two from the group to confirm,
|
|
11
|
+
then fix or mask it once at the source (`MASKING.md`) instead of writing N identical verdicts.
|
|
12
|
+
|
|
13
|
+
## Judge each remaining change
|
|
14
|
+
For each pending entry, in its `judgment` directory:
|
|
15
|
+
|
|
16
|
+
1. Read `request.json`: `regions[]` (bbox, pixelCount, changeType, position — largest first),
|
|
17
|
+
`heuristic`, `summary`, and `paths`.
|
|
18
|
+
2. **Read `locator.png` and `regions.png` together, in one parallel batch** — and batch across
|
|
19
|
+
entries too. `locator.png` is a thumbnail with every region outlined and numbered;
|
|
20
|
+
`regions.png` stacks one `#n expected | #n actual` row per region, in `regions[]` order, with
|
|
21
|
+
context around each change.
|
|
22
|
+
3. When the image dimensions changed there are no regions: read `overview.png` (expected and actual
|
|
23
|
+
scaled side by side, at the same scale) instead. Page height changes are usually intended
|
|
24
|
+
content changes; confirm that the content delta matches a code change.
|
|
25
|
+
4. Only open `paths.expected` / `paths.actual` / `paths.diff` (full size) when a change clearly
|
|
26
|
+
continues outside its crop.
|
|
27
|
+
5. You have the source code — use it. A diff that lines up with a change in the working tree or a
|
|
28
|
+
recent commit is evidence of intent; one with no code change behind it is evidence of noise
|
|
29
|
+
or a regression elsewhere.
|
|
30
|
+
|
|
31
|
+
Labels:
|
|
32
|
+
- `regression-likely` — broken, missing, clipped, overlapping, misaligned, wrong data, truncated
|
|
33
|
+
text, or an unintended side effect of an unrelated change.
|
|
34
|
+
- `intentional-likely` — a deliberate content or style change consistent with the code.
|
|
35
|
+
- `noise-likely` — anti-aliasing, sub-pixel text rendering, non-deterministic content.
|
|
36
|
+
|
|
37
|
+
## Record the verdict
|
|
38
|
+
Preferred — validated, and accepts several names for one shared cause:
|
|
39
|
+
```sh
|
|
40
|
+
cappa agent judge <name> [<name> ...] --label <label> --headline "<one line>" \
|
|
41
|
+
[--rationale "<reason>" ...] [--confidence 0.8] --json
|
|
42
|
+
```
|
|
43
|
+
Or write `verdict.json` next to `request.json`:
|
|
44
|
+
```json
|
|
45
|
+
{ "label": "regression-likely", "headline": "Submit button label is truncated",
|
|
46
|
+
"rationale": ["'Submit' renders as 'Subm' in region #3"], "confidence": 0.9 }
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Then re-run `cappa agent check --json` and continue with step 3 of `SKILL.md`. A verdict whose
|
|
50
|
+
`verdictError` is set was rejected — fix it and re-check.
|
|
51
|
+
|
|
52
|
+
## Shell loops
|
|
53
|
+
Prefer one `cappa agent judge` call with several names over a loop. If you must loop, don't rely on
|
|
54
|
+
word-splitting a variable (zsh, the macOS default, does not split): use
|
|
55
|
+
`while IFS= read -r name; do ...; done <<'EOF'` with one name per line.
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# cappa — masking unstable UI
|
|
2
|
+
|
|
3
|
+
Masking hides elements matching CSS selectors before the screenshot is taken, so content that is
|
|
4
|
+
non-deterministic by nature (clocks, relative times, random avatars, carousels, video, third-party
|
|
5
|
+
iframes, ads) stops failing the diff. Re-approving such a change only postpones the next failure.
|
|
6
|
+
|
|
7
|
+
## Where masks go
|
|
8
|
+
- **Storybook** (`@cappa/plugin-storybook`) — per story, in the story file:
|
|
9
|
+
```ts
|
|
10
|
+
export const Primary = {
|
|
11
|
+
parameters: { cappa: { mask: ['[data-testid="timestamp"]'] } },
|
|
12
|
+
};
|
|
13
|
+
```
|
|
14
|
+
For every story of a component, set it on the default export's `parameters` instead.
|
|
15
|
+
- **Pages** (`@cappa/plugin-pages`) — per page or for all pages via `defaults`:
|
|
16
|
+
```ts
|
|
17
|
+
cappaPluginPages({
|
|
18
|
+
pages: [{ url: '/', name: 'home', mask: ['.cookie-banner'] }],
|
|
19
|
+
defaults: { mask: ['.live-clock'] },
|
|
20
|
+
});
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Picking selectors
|
|
24
|
+
- Prefer stable hooks: `data-testid`, ids, ARIA roles/labels, component class names. Avoid
|
|
25
|
+
`nth-child` and generated class names (`css-1x2y3z`).
|
|
26
|
+
- Mask the smallest element that contains the unstable content, not its whole section — a
|
|
27
|
+
mask also hides any regression inside it.
|
|
28
|
+
- Better than masking, when you can edit the source: make the content deterministic in the story
|
|
29
|
+
(fixed date, seeded data, `args`) or for the capture.
|
|
30
|
+
|
|
31
|
+
## Shared noise across many screenshots
|
|
32
|
+
When many pending judgments share a headline (see `JUDGING.md`), the cause is usually one shared
|
|
33
|
+
element. Fix or mask it once — a shared decorator/preview parameter in Storybook
|
|
34
|
+
(`.storybook/preview` `parameters.cappa.mask`) or `defaults.mask` for pages — then re-capture and
|
|
35
|
+
re-check instead of judging each screenshot. Storybook replaces arrays rather than merging them,
|
|
36
|
+
so a story that sets its own `mask` must repeat the shared selectors.
|
|
37
|
+
|
|
38
|
+
After changing masks, re-capture the affected tasks (`cappa capture --json --filter ...`). The
|
|
39
|
+
masked screenshots will still differ from the unmasked baselines once — approve those after
|
|
40
|
+
confirming with the user.
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: cappa
|
|
3
|
+
description: Run and review Cappa visual regression tests — capture screenshots, judge what changed, fix regressions, approve intended changes. Trigger on "visual test", "visual regression", "screenshot diff", "review screenshots", "cappa", "/cappa".
|
|
4
|
+
metadata:
|
|
5
|
+
opencode/slash: "true"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# cappa
|
|
9
|
+
|
|
10
|
+
Cappa captures screenshots (Storybook stories, pages) with Playwright and diffs them against
|
|
11
|
+
approved baselines. `cappa agent` turns the diff into something you can judge cheaply: a heuristic
|
|
12
|
+
labels each change, and only the ones it cannot settle are handed to you with small crops.
|
|
13
|
+
|
|
14
|
+
Run the CLI as `{{cappa}}` in this project; the examples below write `cappa` for short.
|
|
15
|
+
|
|
16
|
+
Sibling files — read on demand:
|
|
17
|
+
- `JUDGING.md` — judging changes when `pendingJudgments > 0`.
|
|
18
|
+
- `MASKING.md` — silencing non-deterministic UI instead of re-approving it forever.
|
|
19
|
+
|
|
20
|
+
## Be terse
|
|
21
|
+
- Pass `--json` to every `cappa capture` and `cappa agent` call and parse it. Don't echo CLI output back.
|
|
22
|
+
- Don't `ls`/`find`/`cat` for paths the CLI already returns. The full detail of the last check
|
|
23
|
+
(regions, absolute paths, rationale) is in `reportPath` (`<outputDir>/.cappa-agent/report.json`);
|
|
24
|
+
read it only when you need it.
|
|
25
|
+
- Never open full-size screenshots first. Judgment requests come with `locator.png` and
|
|
26
|
+
`regions.png`, typically 10–100x smaller.
|
|
27
|
+
- End with one summary line: `P/T passed · C changed · N new · D deleted` plus one line per
|
|
28
|
+
non-passing screenshot.
|
|
29
|
+
|
|
30
|
+
## 1. Capture
|
|
31
|
+
Run from the directory holding `cappa.config.ts`.
|
|
32
|
+
|
|
33
|
+
```sh
|
|
34
|
+
cappa capture --json
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
- Capture needs whatever the plugins point at to be reachable — read `cappa.config.ts`. For
|
|
38
|
+
`@cappa/plugin-storybook`, the configured Storybook URL must be up (start it with the project's
|
|
39
|
+
storybook script if it is not, and stop it when you are done). For `@cappa/plugin-pages`, the
|
|
40
|
+
page URLs must be reachable.
|
|
41
|
+
- `--json` prints one document to stdout: `{ ok, tasks, screenshots, errors, results, next }`.
|
|
42
|
+
Progress is muted; warnings and errors go to stderr.
|
|
43
|
+
- `errors[]` are tasks that broke (a story threw, a page timed out). Fix those first — there is
|
|
44
|
+
nothing to judge for them.
|
|
45
|
+
- `results[]` are screenshots that differ from their baseline (`changed`, `new`, `deleted`). They
|
|
46
|
+
are what `cappa agent check` reviews.
|
|
47
|
+
- Exit code 1 with an empty `errors[]` just means screenshots differ — expected, not a failure.
|
|
48
|
+
- To re-capture only some screenshots: `cappa capture --json --filter "<glob over task ids>"`.
|
|
49
|
+
Task ids are in `results[].taskId`, e.g. `example-button--primary` for Storybook. A filtered
|
|
50
|
+
capture replaces only the screenshots it captures; everything else keeps its last capture.
|
|
51
|
+
- Skip this step if the user only asked to review what is already captured.
|
|
52
|
+
|
|
53
|
+
## 2. Check
|
|
54
|
+
```sh
|
|
55
|
+
cappa agent check --json
|
|
56
|
+
```
|
|
57
|
+
Returns `{ reportPath, total, passed, changed, new, deleted, pendingJudgments, results }`.
|
|
58
|
+
`results` lists non-passing screenshots only, each
|
|
59
|
+
`{ name, category, verdict?: { label, headline, action, source }, pending?, judgment? }`.
|
|
60
|
+
|
|
61
|
+
- `category: "changed"` carries a `verdict`. `source: "heuristic"` came from the diff signals;
|
|
62
|
+
`source: "agent"` is a verdict you (or another judge) wrote.
|
|
63
|
+
- `pending: true` → the heuristic said `ambiguous`; `judgment` is the directory with the evidence.
|
|
64
|
+
**Read `JUDGING.md`**, judge them, then re-run `cappa agent check --json`.
|
|
65
|
+
- `--judge all` requests a judgment for *every* changed screenshot, not only ambiguous ones — use it
|
|
66
|
+
when the user wants a thorough review or you doubt a heuristic label.
|
|
67
|
+
|
|
68
|
+
## 3. Report and act
|
|
69
|
+
One line per non-passing screenshot: `<name>: <label> — <headline>`. Then, per label:
|
|
70
|
+
|
|
71
|
+
- `regression-likely` → something looks broken. Point the user at it. If they want it fixed (or
|
|
72
|
+
you were asked to fix regressions), find the cause in the working tree (`git diff`, recent
|
|
73
|
+
commits touching the component), fix the code, re-capture just that task with `--filter`, and
|
|
74
|
+
re-check. Never approve a regression.
|
|
75
|
+
- `intentional-likely` → a deliberate restyle or content change. Ask the user to confirm, then
|
|
76
|
+
`cappa agent approve <name> [<name> ...] --json`.
|
|
77
|
+
- `noise-likely` → anti-aliasing or non-deterministic content. Ask once: mask, approve or ignore.
|
|
78
|
+
Prefer masking when the source is inherently unstable (clock, relative time, random avatar,
|
|
79
|
+
animation) — approving only postpones the next failure. See `MASKING.md`.
|
|
80
|
+
- `new` (no baseline yet) → the screenshot is new. Glance at it only if asked; approving makes it
|
|
81
|
+
the baseline.
|
|
82
|
+
- `deleted` (baseline without a capture) → the story/page was removed or failed to capture. Check
|
|
83
|
+
capture output before approving: approving a deleted screenshot **deletes its baseline**.
|
|
84
|
+
|
|
85
|
+
Approving by verdict: `cappa agent approve --label intentional-likely --json` approves every changed
|
|
86
|
+
screenshot whose current verdict has that label.
|
|
87
|
+
|
|
88
|
+
## Hard rules
|
|
89
|
+
- Never approve without explicit user confirmation, and never approve `regression-likely`.
|
|
90
|
+
- `cappa agent approve` matches names exactly. Don't use plain `cappa approve` — its `--filter`
|
|
91
|
+
is a substring match and approves far more than you intend.
|
|
92
|
+
- Never edit files under `expected/`, `actual/`, `diff/` or `.cappa-agent/` by hand (except
|
|
93
|
+
writing `verdict.json`).
|
|
94
|
+
- In CI (`CI=true`), only `cappa agent check` is allowed; baselines change on developer machines.
|
|
95
|
+
- Re-capturing invalidates verdicts for screenshots whose images changed — that is intended.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cappa/cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.14.0",
|
|
4
4
|
"description": "Command line interface for Cappa",
|
|
5
5
|
"author": "Tim Richter",
|
|
6
6
|
"license": "MIT",
|
|
@@ -28,6 +28,7 @@
|
|
|
28
28
|
"files": [
|
|
29
29
|
"dist",
|
|
30
30
|
"bin",
|
|
31
|
+
"integrations",
|
|
31
32
|
"package.json",
|
|
32
33
|
"README.md"
|
|
33
34
|
],
|
|
@@ -35,11 +36,11 @@
|
|
|
35
36
|
"cappa": "./bin/cappa.cjs"
|
|
36
37
|
},
|
|
37
38
|
"dependencies": {
|
|
38
|
-
"@cappa/client": "0.
|
|
39
|
-
"@cappa/config": "0.1.
|
|
40
|
-
"@cappa/core": "0.
|
|
39
|
+
"@cappa/client": "0.10.0",
|
|
40
|
+
"@cappa/config": "0.1.4",
|
|
41
|
+
"@cappa/core": "0.15.0",
|
|
41
42
|
"@cappa/logger": "0.0.12",
|
|
42
|
-
"@cappa/server": "0.
|
|
43
|
+
"@cappa/server": "0.10.0",
|
|
43
44
|
"chalk": "^6.0.0",
|
|
44
45
|
"commander": "15.0.0",
|
|
45
46
|
"picocolors": "1.1.1",
|
|
@@ -50,6 +51,7 @@
|
|
|
50
51
|
"@types/node": "25.9.3",
|
|
51
52
|
"@vitest/coverage-v8": "5.0.0",
|
|
52
53
|
"@vitest/ui": "5.0.0",
|
|
54
|
+
"sharp": "^0.35.4",
|
|
53
55
|
"tsdown": "0.23.0",
|
|
54
56
|
"typescript": "7.0.2",
|
|
55
57
|
"vitest": "5.0.0"
|