@cappa/cli 0.17.1 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.cjs +13 -16
- package/dist/index.cjs.map +1 -1
- package/dist/index.mjs +14 -17
- package/dist/index.mjs.map +1 -1
- package/integrations/opencode/commands/cappa-approve.md +3 -1
- package/integrations/opencode/commands/cappa-review.md +4 -7
- package/integrations/skills/cappa/JUDGING.md +28 -18
- package/integrations/skills/cappa/SKILL.md +17 -26
- package/package.json +5 -5
|
@@ -12,7 +12,9 @@ Current verdicts:
|
|
|
12
12
|
verdicts and stop — ask which ones.
|
|
13
13
|
2. Match what was asked against `results[]`: names must match exactly. A label
|
|
14
14
|
(`intentional-likely`, `noise-likely`) means every changed screenshot with that verdict.
|
|
15
|
-
3.
|
|
15
|
+
3. A changed screenshot still `pending` has not been looked at: offer to judge it first
|
|
16
|
+
(`/cappa-review`), and approve it only if the request names it explicitly.
|
|
17
|
+
Do not approve a `regression-likely` screenshot or a `deleted` one (approving a deletion removes
|
|
16
18
|
its baseline) unless the request names it explicitly — if it does not, list it and ask.
|
|
17
19
|
`cappa agent approve` refuses `regression-likely` on its own; only when the user explicitly
|
|
18
20
|
confirmed that regression, add `--allow-regression`.
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
description: Cappa visual review —
|
|
2
|
+
description: Cappa visual review — judge every changed screenshot, report and offer next steps
|
|
3
3
|
---
|
|
4
4
|
|
|
5
5
|
Run a Cappa visual regression review. Load the `cappa` skill and follow its rules — above all:
|
|
@@ -18,18 +18,15 @@ Current state of the last capture (`{{cappa}} agent check --json`):
|
|
|
18
18
|
2. **Judge** — if `pendingJudgments > 0`, hand the pending screenshots to the `cappa-judge`
|
|
19
19
|
subagent (Task tool) instead of reading the images yourself; it keeps the crops out of this
|
|
20
20
|
conversation and records a verdict per screenshot. Then re-run `{{cappa}} agent check --json`.
|
|
21
|
+
Every changed screenshot is pending until judged — hand all of them over, in batches as the
|
|
22
|
+
skill's `JUDGING.md` describes.
|
|
21
23
|
If the judge replies that it cannot read images, pass its message on to the user and stop
|
|
22
24
|
there — never judge the screenshots yourself without seeing them.
|
|
23
|
-
For a thorough second opinion on every change — or when you doubt a heuristic label, e.g. a
|
|
24
|
-
`regression-likely` that matches a deliberate code change — run
|
|
25
|
-
`{{cappa}} agent check --judge all --json` first so every changed screenshot gets a request,
|
|
26
|
-
and have the subagent judge all of them (requests stay pending until judged).
|
|
27
25
|
3. **Report** — one line per non-passing screenshot, grouped by label:
|
|
28
26
|
`<name>: <label> — <headline>`, then `new` and `deleted` screenshots. End with
|
|
29
27
|
`P/T passed · C changed · N new · D deleted`.
|
|
30
28
|
4. **Offer next steps**, and wait for the user:
|
|
31
|
-
- `regression-likely` →
|
|
32
|
-
Then offer to find and fix the cause, re-capture just those tasks with
|
|
29
|
+
- `regression-likely` → offer to find and fix the cause, re-capture just those tasks with
|
|
33
30
|
`{{cappa}} capture --json --filter "<taskId>"`, and review again.
|
|
34
31
|
- `intentional-likely` → ask to approve them: `/cappa-approve <names>`.
|
|
35
32
|
- `noise-likely` → offer to mask the element (the skill's `MASKING.md`) or approve.
|
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
# cappa — judging changes
|
|
2
2
|
|
|
3
|
-
Used when `cappa agent check --json` reports `pendingJudgments > 0`:
|
|
4
|
-
|
|
3
|
+
Used when `cappa agent check --json` reports `pendingJudgments > 0`: every changed screenshot
|
|
4
|
+
waits for a judge, and nothing has labelled it before you look. Cappa's diff threshold already
|
|
5
|
+
filters sub-pixel rendering, so assume each change is real until the images show otherwise.
|
|
5
6
|
|
|
6
7
|
## Who judges
|
|
7
8
|
If the `cappa-judge` subagent is installed (`cappa agent init` adds it for Claude Code and
|
|
@@ -21,22 +22,24 @@ hand the `judgment` directories to any subagent whose model can. Never substitut
|
|
|
21
22
|
tooling (no ImageMagick, no manual crops, no pixel statistics) and never judge blind.
|
|
22
23
|
|
|
23
24
|
## Judge everything that is pending
|
|
24
|
-
A request stays pending through every later check until it has a verdict
|
|
25
|
-
|
|
26
|
-
|
|
25
|
+
A request stays pending through every later check until it has a verdict, so `pendingJudgments`
|
|
26
|
+
only reaches 0 when every changed screenshot is judged. Several screenshots with one shared cause
|
|
27
|
+
take one `cappa agent judge` call.
|
|
27
28
|
|
|
28
29
|
## Look for one shared cause first
|
|
29
|
-
Before opening anything, group the pending `results[]` by `
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
30
|
+
Before opening anything, group the pending `results[]` by `change`. It is built so that the same
|
|
31
|
+
change produces the same text — `1 region @ bottom-right (0.02%)` across many screenshots is
|
|
32
|
+
likely one shared element (footer date, version badge, theme toggle, a shared component everyone
|
|
33
|
+
renders). Open two or three from the group to confirm they really show the same change, then fix
|
|
34
|
+
or mask it once at the source (`MASKING.md`) instead of writing N identical verdicts. A matching
|
|
35
|
+
`change` line is a hint, not evidence: judge every screenshot you have not confirmed.
|
|
34
36
|
|
|
35
37
|
## Judge each remaining change
|
|
36
38
|
For each pending entry, in its `judgment` directory:
|
|
37
39
|
|
|
38
|
-
1. Read `request.json`: `regions[]` (bbox, pixelCount,
|
|
39
|
-
|
|
40
|
+
1. Read `request.json`: `change`, `regions[]` (bbox, pixelCount, position, shape — largest
|
|
41
|
+
first), `dimensions` and `paths`. It says where pixels differ, not what kind of change it is —
|
|
42
|
+
that is for you to see.
|
|
40
43
|
2. **Read `locator.png` and `regions.png` together, in one parallel batch** — and batch across
|
|
41
44
|
entries too. `locator.png` is a thumbnail with every region outlined and numbered;
|
|
42
45
|
`regions.png` stacks one `#n expected | #n actual` row per region, in `regions[]` order, with
|
|
@@ -47,18 +50,25 @@ For each pending entry, in its `judgment` directory:
|
|
|
47
50
|
top-left — in a crop, area only one image covers is empty. With no regions, only the size
|
|
48
51
|
changed (whitespace grew or shrank). Size changes are usually intended content changes;
|
|
49
52
|
confirm that the content delta matches a code change.
|
|
50
|
-
4.
|
|
53
|
+
4. Compare sizes and positions, not just content. A font that is larger, smaller or bolder,
|
|
54
|
+
text that wraps or truncates differently, and padding, gaps or alignment that moved are easy
|
|
55
|
+
to dismiss as rendering noise in a crop — check glyph heights, line counts and edges against
|
|
56
|
+
each other in the `expected | actual` pair.
|
|
57
|
+
5. Only open `paths.expected` / `paths.actual` / `paths.diff` (full size) when a change clearly
|
|
51
58
|
continues outside its crop.
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
59
|
+
6. Then use the source code. A change that lines up with a change in the working tree or a recent
|
|
60
|
+
commit is evidence of intent — but only for what that code change should do: a button
|
|
61
|
+
restyle that also shrinks the heading next to it is a regression. A change with no code
|
|
62
|
+
change behind it is a regression elsewhere or non-deterministic content. Never let the code
|
|
63
|
+
diff stand in for looking.
|
|
56
64
|
|
|
57
65
|
Labels:
|
|
58
66
|
- `regression-likely` — broken, missing, clipped, overlapping, misaligned, wrong data, truncated
|
|
59
67
|
text, or an unintended side effect of an unrelated change.
|
|
60
68
|
- `intentional-likely` — a deliberate content or style change consistent with the code.
|
|
61
|
-
- `noise-likely` —
|
|
69
|
+
- `noise-likely` — non-deterministic content (clock, random data, animation frame) or rendering
|
|
70
|
+
differences you can see are invisible to a user. Rare: the diff threshold already filters
|
|
71
|
+
sub-pixel rendering, so do not reach for this label because a change is small.
|
|
62
72
|
|
|
63
73
|
## Record the verdict
|
|
64
74
|
With `cappa agent judge` — it validates the verdict, and takes several names for one shared
|
|
@@ -8,14 +8,16 @@ metadata:
|
|
|
8
8
|
# cappa
|
|
9
9
|
|
|
10
10
|
Cappa captures screenshots (Storybook stories, pages) with Playwright and diffs them against
|
|
11
|
-
approved baselines. `cappa agent` turns the diff into something you can judge cheaply:
|
|
12
|
-
|
|
11
|
+
approved baselines. `cappa agent` turns the diff into something you can judge cheaply: every
|
|
12
|
+
changed screenshot gets a judgment request with small crops of what changed, and a judge looks at
|
|
13
|
+
each one. Cappa's diff threshold already absorbs sub-pixel rendering, so a `changed` screenshot
|
|
14
|
+
differs by more than the project tolerates — none is dismissed without being looked at.
|
|
13
15
|
|
|
14
16
|
Run the CLI as `{{cappa}}` from the project root — it already runs in the package holding
|
|
15
17
|
`cappa.config.ts`, so don't `cd` first. The examples below write `cappa` for short.
|
|
16
18
|
|
|
17
19
|
Sibling files — read on demand:
|
|
18
|
-
- `JUDGING.md` — judging changes
|
|
20
|
+
- `JUDGING.md` — judging changes (whenever `pendingJudgments > 0`).
|
|
19
21
|
- `MASKING.md` — silencing non-deterministic UI instead of re-approving it forever.
|
|
20
22
|
|
|
21
23
|
## Be terse
|
|
@@ -54,36 +56,25 @@ cappa agent check --json
|
|
|
54
56
|
```
|
|
55
57
|
Returns `{ reportPath, total, passed, changed, new, deleted, pendingJudgments, results }`.
|
|
56
58
|
`results` lists non-passing screenshots only, each
|
|
57
|
-
`{ name, category, verdict?: { label, headline, action
|
|
59
|
+
`{ name, category, change?, verdict?: { label, headline, action }, pending?, judgment? }`.
|
|
58
60
|
It exits 1 whenever anything is not passing — that is the normal result, not a failure. A real
|
|
59
61
|
failure prints `{ "error": "..." }` instead.
|
|
60
62
|
|
|
61
|
-
- `
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
- `pending: true` → the heuristic said `ambiguous`; `judgment` is the directory with the evidence.
|
|
69
|
-
**Read `JUDGING.md`**, judge them, then re-run `cappa agent check --json`.
|
|
70
|
-
- `--judge all` requests a judgment for *every* changed screenshot, not only ambiguous ones, so
|
|
71
|
-
each gets a `judgment` directory with `locator.png` and `regions.png`. Use it whenever you need
|
|
72
|
-
to look anyway: you doubt a heuristic label, or the user wants a thorough review. Then judge per
|
|
73
|
-
`JUDGING.md` like any other pending entry.
|
|
74
|
-
- Requests are sticky: once made, a request stays `pending` through every later check until it
|
|
75
|
-
gets a verdict, the screenshot is approved, or a re-capture changes its images. After
|
|
76
|
-
`--judge all`, judge every pending entry — including the ones whose heuristic label you agree
|
|
77
|
-
with (one `cappa agent judge` call per shared label is enough) — or `pendingJudgments` never
|
|
78
|
-
reaches 0.
|
|
63
|
+
- Every `changed` screenshot is `pending: true` until a judge records a verdict; `judgment` is the
|
|
64
|
+
directory with the evidence. **Read `JUDGING.md`**, judge all of them, then re-run
|
|
65
|
+
`cappa agent check --json` until `pendingJudgments` is 0.
|
|
66
|
+
- `change` says only where and how much changed (`2 regions @ top-left (1.20%)`), never what
|
|
67
|
+
kind of change it is. Screenshots with the same `change` likely share one cause.
|
|
68
|
+
- A `verdict` is always a judge's, made after looking at the images. It stays applied until the
|
|
69
|
+
screenshot is approved or a re-capture changes its images; then it is pending again.
|
|
79
70
|
- Judging means reading images. When the `cappa-judge` subagent is installed, it always does the
|
|
80
|
-
judging, not you — see `JUDGING.md`. Never judge blind
|
|
71
|
+
judging, not you — see `JUDGING.md`. Never judge blind, and never label a change from its
|
|
72
|
+
`change` line, its pixel count or the code diff alone.
|
|
81
73
|
|
|
82
74
|
## 3. Report and act
|
|
83
75
|
One line per non-passing screenshot: `<name>: <label> — <headline>`. Then, per label:
|
|
84
76
|
|
|
85
|
-
- `regression-likely` → something looks broken.
|
|
86
|
-
(`source: "heuristic"`), confirm it against the code first (see step 2). Point the user at it.
|
|
77
|
+
- `regression-likely` → something looks broken. Point the user at it.
|
|
87
78
|
If they want it fixed (or you were asked to fix regressions), find the cause in the working
|
|
88
79
|
tree (`git diff`, recent commits touching the component), fix the code, re-capture just that
|
|
89
80
|
task with `--filter`, and re-check. Never approve a regression.
|
|
@@ -111,6 +102,6 @@ screenshot whose current verdict has that label.
|
|
|
111
102
|
- Never edit files under `expected/`, `actual/`, `diff/` or `.cappa-agent/` by hand — verdicts
|
|
112
103
|
go through `cappa agent judge`.
|
|
113
104
|
- Never hand-roll crops, image diffs or pixel forensics (ImageMagick, scripts). Cappa's crops are
|
|
114
|
-
the evidence;
|
|
105
|
+
the evidence; `cappa agent check --json` writes them for every changed screenshot.
|
|
115
106
|
- In CI (`CI=true`), only `cappa agent check` is allowed; baselines change on developer machines.
|
|
116
107
|
- Re-capturing invalidates verdicts for screenshots whose images changed — that is intended.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cappa/cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.18.0",
|
|
4
4
|
"description": "Command line interface for Cappa",
|
|
5
5
|
"author": "Tim Richter",
|
|
6
6
|
"license": "MIT",
|
|
@@ -37,10 +37,10 @@
|
|
|
37
37
|
},
|
|
38
38
|
"dependencies": {
|
|
39
39
|
"@cappa/client": "0.10.0",
|
|
40
|
-
"@cappa/config": "0.3.
|
|
41
|
-
"@cappa/core": "0.
|
|
40
|
+
"@cappa/config": "0.3.1",
|
|
41
|
+
"@cappa/core": "0.18.0",
|
|
42
42
|
"@cappa/logger": "0.0.12",
|
|
43
|
-
"@cappa/server": "0.10.
|
|
43
|
+
"@cappa/server": "0.10.3",
|
|
44
44
|
"chalk": "^6.0.0",
|
|
45
45
|
"commander": "15.0.0",
|
|
46
46
|
"picocolors": "1.1.1",
|
|
@@ -50,7 +50,7 @@
|
|
|
50
50
|
"@arethetypeswrong/cli": "0.18.5",
|
|
51
51
|
"@types/node": "25.9.3",
|
|
52
52
|
"@vitest/coverage-v8": "5.0.0",
|
|
53
|
-
"@vitest/ui": "5.0.
|
|
53
|
+
"@vitest/ui": "5.0.5",
|
|
54
54
|
"sharp": "^0.35.4",
|
|
55
55
|
"tsdown": "0.23.0",
|
|
56
56
|
"typescript": "7.0.2",
|