@cappa/cli 0.13.3 → 0.15.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,14 @@
1
+ ---
2
+ description: Judge pending Cappa screenshot changes in a sandboxed subagent
3
+ agent: cappa-judge
4
+ subtask: true
5
+ ---
6
+
7
+ Judge the pending Cappa screenshot changes.
8
+
9
+ Only these screenshots, if any are named: $ARGUMENTS
10
+ (If none are named, judge every result with `pending: true`.)
11
+
12
+ Current state:
13
+
14
+ !`{{cappa}} agent check --json || true`
@@ -0,0 +1,34 @@
1
+ ---
2
+ description: Cappa visual review — label changed screenshots, judge the ambiguous ones, report and offer next steps
3
+ ---
4
+
5
+ Run a Cappa visual regression review. Load the `cappa` skill and follow its rules — above all:
6
+ never approve without the user's confirmation, and never approve a `regression-likely` change.
7
+
8
+ Request: $ARGUMENTS
9
+
10
+ Current state of the last capture (`{{cappa}} agent check --json`):
11
+
12
+ !`{{cappa}} agent check --json || true`
13
+
14
+ ## Flow
15
+ 1. **Capture first** when the request asks for it, or when the state above has `total: 0`: run
16
+ `{{cappa}} capture --json` (start whatever `cappa.config.ts` points at first, per the skill).
17
+ Fix anything in its `errors[]` before reviewing. Then re-run `{{cappa}} agent check --json`.
18
+ 2. **Judge** — if `pendingJudgments > 0`, hand the pending screenshots to the `cappa-judge`
19
+ subagent (Task tool) instead of reading the images yourself; it keeps the crops out of this
20
+ conversation and records a verdict per screenshot. Then re-run `{{cappa}} agent check --json`.
21
+ For a thorough second opinion on every change — or when you doubt a heuristic label, e.g. a
22
+ `regression-likely` that matches a deliberate code change — run
23
+ `{{cappa}} agent check --judge all --json` first so every changed screenshot gets a request,
24
+ and have the subagent judge all of them (requests stay pending until judged).
25
+ 3. **Report** — one line per non-passing screenshot, grouped by label:
26
+ `<name>: <label> — <headline>`, then `new` and `deleted` screenshots. End with
27
+ `P/T passed · C changed · N new · D deleted`.
28
+ 4. **Offer next steps**, and wait for the user:
29
+ - `regression-likely` → if the verdict is the heuristic's, check it against the code first.
30
+ Then offer to find and fix the cause, re-capture just those tasks with
31
+ `{{cappa}} capture --json --filter "<taskId>"`, and review again.
32
+ - `intentional-likely` → ask to approve them: `/cappa-approve <names>`.
33
+ - `noise-likely` → offer to mask the element (the skill's `MASKING.md`) or approve.
34
+ - The verdicts are also visible in the review UI (`{{cappa}} review`).
@@ -0,0 +1,68 @@
1
+ # cappa — judging changes
2
+
3
+ Used when `cappa agent check --json` reports `pendingJudgments > 0`: the heuristic labelled those
4
+ changes `ambiguous` and is deferring to you — or you asked for every change with `--judge all`.
5
+
6
+ ## Can you read images?
7
+ Judging is visual. If your model cannot read images, don't substitute your own tooling (no
8
+ ImageMagick, no manual crops, no pixel statistics). Hand the pending `judgment` directories to a
9
+ subagent whose model can read images — the `cappa-judge` subagent if `cappa agent init` installed
10
+ one — and have it follow this file; it records verdicts with `cappa agent judge` (or returns
11
+ labels and rationale for you to record).
12
+
13
+ ## Judge everything that is pending
14
+ A request stays pending through every later check until it has a verdict. After `--judge all`
15
+ that includes changes whose heuristic label you agree with — record those too (one call per
16
+ shared label), or `pendingJudgments` never reaches 0.
17
+
18
+ ## Look for one shared cause first
19
+ Before opening anything, group the pending `results[]` by `verdict.headline`. Headlines are built so
20
+ that the same kind of change produces the same text — `1 content-change @ bottom-right (0.02%, low)`
21
+ across many screenshots is almost certainly one shared element (footer date, version badge,
22
+ theme toggle, a shared component everyone renders). Open one or two from the group to confirm,
23
+ then fix or mask it once at the source (`MASKING.md`) instead of writing N identical verdicts.
24
+
25
+ ## Judge each remaining change
26
+ For each pending entry, in its `judgment` directory:
27
+
28
+ 1. Read `request.json`: `regions[]` (bbox, pixelCount, changeType, position — largest first),
29
+ `heuristic`, `summary`, and `paths`.
30
+ 2. **Read `locator.png` and `regions.png` together, in one parallel batch** — and batch across
31
+ entries too. `locator.png` is a thumbnail with every region outlined and numbered;
32
+ `regions.png` stacks one `#n expected | #n actual` row per region, in `regions[]` order, with
33
+ context around each change.
34
+ 3. When the image dimensions changed there are no regions: read `overview.png` (expected and actual
35
+ scaled side by side, at the same scale) instead. Page height changes are usually intended
36
+ content changes; confirm that the content delta matches a code change.
37
+ 4. Only open `paths.expected` / `paths.actual` / `paths.diff` (full size) when a change clearly
38
+ continues outside its crop.
39
+ 5. You have the source code — use it. A diff that lines up with a change in the working tree or a
40
+ recent commit is evidence of intent; one with no code change behind it is evidence of noise
41
+ or a regression elsewhere. The heuristic's label is one more signal, not the answer: it calls
42
+ structural changes `regression-likely` even when they were made on purpose.
43
+
44
+ Labels:
45
+ - `regression-likely` — broken, missing, clipped, overlapping, misaligned, wrong data, truncated
46
+ text, or an unintended side effect of an unrelated change.
47
+ - `intentional-likely` — a deliberate content or style change consistent with the code.
48
+ - `noise-likely` — anti-aliasing, sub-pixel text rendering, non-deterministic content.
49
+
50
+ ## Record the verdict
51
+ Preferred — validated, and accepts several names for one shared cause:
52
+ ```sh
53
+ cappa agent judge <name> [<name> ...] --label <label> --headline "<one line>" \
54
+ [--rationale "<reason>" ...] [--confidence 0.8] --json
55
+ ```
56
+ Or write `verdict.json` next to `request.json`:
57
+ ```json
58
+ { "label": "regression-likely", "headline": "Submit button label is truncated",
59
+ "rationale": ["'Submit' renders as 'Subm' in region #3"], "confidence": 0.9 }
60
+ ```
61
+
62
+ Then re-run `cappa agent check --json` and continue with step 3 of `SKILL.md`. A verdict whose
63
+ `verdictError` is set was rejected — fix it and re-check.
64
+
65
+ ## Shell loops
66
+ Prefer one `cappa agent judge` call with several names over a loop. If you must loop, don't rely on
67
+ word-splitting a variable (zsh, the macOS default, does not split): use
68
+ `while IFS= read -r name; do ...; done <<'EOF'` with one name per line.
@@ -0,0 +1,40 @@
1
+ # cappa — masking unstable UI
2
+
3
+ Masking hides elements matching CSS selectors before the screenshot is taken, so content that is
4
+ non-deterministic by nature (clocks, relative times, random avatars, carousels, video, third-party
5
+ iframes, ads) stops failing the diff. Re-approving such a change only postpones the next failure.
6
+
7
+ ## Where masks go
8
+ - **Storybook** (`@cappa/plugin-storybook`) — per story, in the story file:
9
+ ```ts
10
+ export const Primary = {
11
+ parameters: { cappa: { mask: ['[data-testid="timestamp"]'] } },
12
+ };
13
+ ```
14
+ For every story of a component, set it on the default export's `parameters` instead.
15
+ - **Pages** (`@cappa/plugin-pages`) — per page or for all pages via `defaults`:
16
+ ```ts
17
+ cappaPluginPages({
18
+ pages: [{ url: '/', name: 'home', mask: ['.cookie-banner'] }],
19
+ defaults: { mask: ['.live-clock'] },
20
+ });
21
+ ```
22
+
23
+ ## Picking selectors
24
+ - Prefer stable hooks: `data-testid`, ids, ARIA roles/labels, component class names. Avoid
25
+ `nth-child` and generated class names (`css-1x2y3z`).
26
+ - Mask the smallest element that contains the unstable content, not its whole section — a
27
+ mask also hides any regression inside it.
28
+ - Better than masking, when you can edit the source: make the content deterministic in the story
29
+ (fixed date, seeded data, `args`) or for the capture.
30
+
31
+ ## Shared noise across many screenshots
32
+ When many pending judgments share a headline (see `JUDGING.md`), the cause is usually one shared
33
+ element. Fix or mask it once — a shared decorator/preview parameter in Storybook
34
+ (`.storybook/preview` `parameters.cappa.mask`) or `defaults.mask` for pages — then re-capture and
35
+ re-check instead of judging each screenshot. Storybook replaces arrays rather than merging them,
36
+ so a story that sets its own `mask` must repeat the shared selectors.
37
+
38
+ After changing masks, re-capture the affected tasks (`cappa capture --json --filter ...`). The
39
+ masked screenshots will still differ from the unmasked baselines once — approve those after
40
+ confirming with the user.
@@ -0,0 +1,120 @@
1
+ ---
2
+ name: cappa
3
+ description: Run and review Cappa visual regression tests — capture screenshots, judge what changed, fix regressions, approve intended changes. Trigger on "visual test", "visual regression", "screenshot diff", "review screenshots", "cappa", "/cappa".
4
+ metadata:
5
+ opencode/slash: "true"
6
+ ---
7
+
8
+ # cappa
9
+
10
+ Cappa captures screenshots (Storybook stories, pages) with Playwright and diffs them against
11
+ approved baselines. `cappa agent` turns the diff into something you can judge cheaply: a heuristic
12
+ labels each change, and only the ones it cannot settle are handed to you with small crops.
13
+
14
+ Run the CLI as `{{cappa}}` in this project; the examples below write `cappa` for short.
15
+
16
+ Sibling files — read on demand:
17
+ - `JUDGING.md` — judging changes when `pendingJudgments > 0`.
18
+ - `MASKING.md` — silencing non-deterministic UI instead of re-approving it forever.
19
+
20
+ ## Be terse
21
+ - Pass `--json` to every `cappa capture` and `cappa agent` call and parse it. Don't echo CLI output back.
22
+ - Don't `ls`/`find`/`cat` for paths the CLI already returns. The full detail of the last check
23
+ (regions, absolute paths, rationale) is in `reportPath` (`<outputDir>/.cappa-agent/report.json`);
24
+ read it only when you need it.
25
+ - Never open full-size screenshots first. Judgment requests come with `locator.png` and
26
+ `regions.png`, typically 10–100x smaller.
27
+ - End with one summary line: `P/T passed · C changed · N new · D deleted` plus one line per
28
+ non-passing screenshot.
29
+
30
+ ## 1. Capture
31
+ Run from the directory holding `cappa.config.ts`.
32
+
33
+ ```sh
34
+ cappa capture --json
35
+ ```
36
+
37
+ - Capture needs whatever the plugins point at to be reachable — read `cappa.config.ts`. For
38
+ `@cappa/plugin-storybook`, the configured Storybook URL must be up (start it with the project's
39
+ storybook script if it is not, and stop it when you are done). For `@cappa/plugin-pages`, the
40
+ page URLs must be reachable.
41
+ - `--json` prints one document to stdout: `{ ok, tasks, screenshots, errors, results, next }`.
42
+ Progress is muted; warnings and errors go to stderr.
43
+ - `errors[]` are tasks that broke (a story threw, a page timed out). Fix those first — there is
44
+ nothing to judge for them.
45
+ - `results[]` are screenshots that differ from their baseline (`changed`, `new`, `deleted`). They
46
+ are what `cappa agent check` reviews.
47
+ - Exit code 1 with an empty `errors[]` just means screenshots differ — expected, not a failure.
48
+ - To re-capture only some screenshots: `cappa capture --json --filter "<glob over task ids>"`.
49
+ Task ids are in `results[].taskId`, e.g. `example-button--primary` for Storybook. A filtered
50
+ capture replaces only the screenshots it captures; everything else keeps its last capture.
51
+ - Skip this step if the user only asked to review what is already captured.
52
+
53
+ ## 2. Check
54
+ ```sh
55
+ cappa agent check --json
56
+ ```
57
+ Returns `{ reportPath, total, passed, changed, new, deleted, pendingJudgments, results }`.
58
+ `results` lists non-passing screenshots only, each
59
+ `{ name, category, verdict?: { label, headline, action, source }, pending?, judgment? }`.
60
+ It exits 1 whenever anything is not passing — that is the normal result, not a failure. A real
61
+ failure prints `{ "error": "..." }` instead.
62
+
63
+ - `category: "changed"` carries a `verdict`. `source: "heuristic"` came from the diff signals;
64
+ `source: "agent"` is a verdict you (or another judge) wrote.
65
+ - A heuristic label is a guess from pixel signals, not a finding. In particular it calls any
66
+ medium- or high-severity structural change `regression-likely` — including content that was
67
+ added or removed on purpose. Before treating a heuristic `regression-likely` as broken, check it
68
+ against the code; if a deliberate change explains it, judge it (`--judge all`, below) instead of
69
+ "fixing" the feature.
70
+ - `pending: true` → the heuristic said `ambiguous`; `judgment` is the directory with the evidence.
71
+ **Read `JUDGING.md`**, judge them, then re-run `cappa agent check --json`.
72
+ - `--judge all` requests a judgment for *every* changed screenshot, not only ambiguous ones, so
73
+ each gets a `judgment` directory with `locator.png` and `regions.png`. Use it whenever you need
74
+ to look anyway: you doubt a heuristic label, or the user wants a thorough review. Then judge per
75
+ `JUDGING.md` like any other pending entry.
76
+ - Requests are sticky: once made, a request stays `pending` through every later check until it
77
+ gets a verdict, the screenshot is approved, or a re-capture changes its images. After
78
+ `--judge all`, judge every pending entry — including the ones whose heuristic label you agree
79
+ with (one `cappa agent judge` call per shared label is enough) — or `pendingJudgments` never
80
+ reaches 0.
81
+ - Judging means reading images. If you cannot read images yourself, don't improvise — hand the
82
+ `judgment` directories to a subagent whose model can (the `cappa-judge` subagent, when
83
+ `cappa agent init` installed one), let it judge per `JUDGING.md`, and use the verdicts it
84
+ records (or run `cappa agent judge` with the labels it returns).
85
+
86
+ ## 3. Report and act
87
+ One line per non-passing screenshot: `<name>: <label> — <headline>`. Then, per label:
88
+
89
+ - `regression-likely` → something looks broken. If the verdict is the heuristic's
90
+ (`source: "heuristic"`), confirm it against the code first (see step 2). Point the user at it.
91
+ If they want it fixed (or you were asked to fix regressions), find the cause in the working
92
+ tree (`git diff`, recent commits touching the component), fix the code, re-capture just that
93
+ task with `--filter`, and re-check. Never approve a regression.
94
+ - `intentional-likely` → a deliberate restyle or content change. Ask the user to confirm, then
95
+ `cappa agent approve <name> [<name> ...] --json`.
96
+ - `noise-likely` → anti-aliasing or non-deterministic content. Ask once: mask, approve or ignore.
97
+ Prefer masking when the source is inherently unstable (clock, relative time, random avatar,
98
+ animation) — approving only postpones the next failure. See `MASKING.md`.
99
+ - `new` (no baseline yet) → the screenshot is new. There are no crops — looking means reading the
100
+ full-size image in `<outputDir>/actual/<name>.png`, so do it only if asked (and through an
101
+ image-capable subagent if you cannot read images). Approving makes it the baseline.
102
+ - `deleted` (baseline without a capture) → the story/page was removed or failed to capture. Check
103
+ capture output before approving: approving a deleted screenshot **deletes its baseline**.
104
+
105
+ Approving by verdict: `cappa agent approve --label intentional-likely --json` approves every changed
106
+ screenshot whose current verdict has that label.
107
+
108
+ ## Hard rules
109
+ - Never approve without explicit user confirmation, and never approve `regression-likely`.
110
+ `cappa agent approve` refuses to; if the change is in fact intended, judge it
111
+ `intentional-likely` first. Pass `--allow-regression` only when the user explicitly tells you to
112
+ approve that regression.
113
+ - `cappa agent approve` matches names exactly. Don't use plain `cappa approve` — its `--filter`
114
+ is a substring match and approves far more than you intend.
115
+ - Never edit files under `expected/`, `actual/`, `diff/` or `.cappa-agent/` by hand (except
116
+ writing `verdict.json`).
117
+ - Never hand-roll crops, image diffs or pixel forensics (ImageMagick, scripts). Cappa's crops are
118
+ the evidence; if a screenshot has none, `cappa agent check --json --judge all` produces them.
119
+ - In CI (`CI=true`), only `cappa agent check` is allowed; baselines change on developer machines.
120
+ - Re-capturing invalidates verdicts for screenshots whose images changed — that is intended.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@cappa/cli",
3
- "version": "0.13.3",
3
+ "version": "0.15.0",
4
4
  "description": "Command line interface for Cappa",
5
5
  "author": "Tim Richter",
6
6
  "license": "MIT",
@@ -28,6 +28,7 @@
28
28
  "files": [
29
29
  "dist",
30
30
  "bin",
31
+ "integrations",
31
32
  "package.json",
32
33
  "README.md"
33
34
  ],
@@ -35,11 +36,11 @@
35
36
  "cappa": "./bin/cappa.cjs"
36
37
  },
37
38
  "dependencies": {
38
- "@cappa/client": "0.9.0",
39
- "@cappa/config": "0.1.2",
40
- "@cappa/core": "0.13.2",
39
+ "@cappa/client": "0.10.0",
40
+ "@cappa/config": "0.1.4",
41
+ "@cappa/core": "0.15.0",
41
42
  "@cappa/logger": "0.0.12",
42
- "@cappa/server": "0.9.3",
43
+ "@cappa/server": "0.10.0",
43
44
  "chalk": "^6.0.0",
44
45
  "commander": "15.0.0",
45
46
  "picocolors": "1.1.1",
@@ -50,6 +51,7 @@
50
51
  "@types/node": "25.9.3",
51
52
  "@vitest/coverage-v8": "5.0.0",
52
53
  "@vitest/ui": "5.0.0",
54
+ "sharp": "^0.35.4",
53
55
  "tsdown": "0.23.0",
54
56
  "typescript": "7.0.2",
55
57
  "vitest": "5.0.0"