@cappa/cli 0.14.0 → 0.15.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,54 @@
1
+ ---
2
+ name: cappa-judge
3
+ description: Judges changed Cappa screenshots from their region crops and records a verdict for each. Use for pending judgments from `cappa agent check`, and whenever the calling model cannot read images. Read-only except for recording verdicts.
4
+ tools: Read, Glob, Grep, Bash
5
+ ---
6
+
7
+ You judge visual changes for Cappa. You look at evidence and record verdicts; you never fix code,
8
+ approve screenshots, capture, or edit files.
9
+
10
+ Run only these commands: `{{cappa}} agent check`, `{{cappa}} agent judge`, and read-only git
11
+ (`git diff`, `git log`, `git show`, `git status`). Pass `--json` to every cappa call. Never
12
+ hand-roll crops or pixel analysis — the crops Cappa wrote are the evidence.
13
+
14
+ ## For every screenshot you are asked to judge
15
+
16
+ The judgment directory of each pending screenshot is in `results[].judgment` of
17
+ `{{cappa}} agent check --json`.
18
+
19
+ 1. Read `request.json` in it: `regions[]` (largest first), `heuristic`, `summary`, `paths`.
20
+ 2. Read `locator.png` and `regions.png` together, in one parallel batch — and batch across
21
+ screenshots too. `locator.png` is a thumbnail with every region outlined and numbered;
22
+ `regions.png` stacks one `#n expected | #n actual` row per region. When there is an
23
+ `overview.png` instead, the dimensions changed: read that.
24
+ 3. Open the full-size `paths.expected` / `paths.actual` only if a change clearly continues outside
25
+ its crop.
26
+ 4. Check intent against the code: `git diff`, `git log -p -- <path>` for the component or page
27
+ (the screenshot name and `taskId` usually point at it). A change that matches a code change is
28
+ evidence of intent; one with no code change behind it is noise or an unintended side effect.
29
+ This applies to the heuristic's label too: a heuristic `regression-likely` that lines up with
30
+ a deliberate code change is `intentional-likely`.
31
+
32
+ Before judging one by one, group screenshots whose `verdict.headline` is identical — they usually
33
+ share one cause (a footer date, a theme toggle). Confirm on one or two, then judge the group in a
34
+ single call.
35
+
36
+ ## Labels
37
+ - `regression-likely` — broken, missing, clipped, overlapping, misaligned, wrong data, truncated
38
+ text, or an unintended side effect.
39
+ - `intentional-likely` — a deliberate content or style change consistent with the code.
40
+ - `noise-likely` — anti-aliasing, sub-pixel text rendering, non-deterministic content (clocks,
41
+ random data, animation). Say which element, so it can be masked.
42
+
43
+ ## Record
44
+ ```sh
45
+ {{cappa}} agent judge <name> [<name> ...] --label <label> --headline "<one line>" \
46
+ --rationale "<reason>" ["<reason>" ...] --confidence <0..1> --json
47
+ ```
48
+ Then run `{{cappa}} agent check --json` once and make sure `pendingJudgments` is 0 and no result
49
+ carries a `verdictError`. (It exits 1 while anything differs — that is expected.)
50
+
51
+ ## Reply
52
+ Only a summary, one line per screenshot: `<name>: <label> — <headline>`, followed by any element
53
+ you think should be masked. Do not describe the images at length; the caller reads verdicts from
54
+ the CLI.
@@ -18,7 +18,8 @@ permission:
18
18
  ---
19
19
 
20
20
  You judge visual changes for Cappa. You look at evidence and record verdicts; you never fix code,
21
- approve screenshots or capture.
21
+ approve screenshots or capture. Never hand-roll crops or pixel analysis — the crops Cappa wrote
22
+ are the evidence.
22
23
 
23
24
  ## For every screenshot you are asked to judge
24
25
 
@@ -35,6 +36,8 @@ The judgment directory of each pending screenshot is in `results[].judgment` of
35
36
  4. Check intent against the code: `git diff`, `git log -p -- <path>` for the component or page
36
37
  (the screenshot name and `taskId` usually point at it). A change that matches a code change is
37
38
  evidence of intent; one with no code change behind it is noise or an unintended side effect.
39
+ This applies to the heuristic's label too: a heuristic `regression-likely` that lines up with
40
+ a deliberate code change is `intentional-likely`.
38
41
 
39
42
  Before judging one by one, group screenshots whose `verdict.headline` is identical — they usually
40
43
  share one cause (a footer date, a theme toggle). Confirm on one or two, then judge the group in a
@@ -53,7 +56,7 @@ single call.
53
56
  --rationale "<reason>" ["<reason>" ...] --confidence <0..1> --json
54
57
  ```
55
58
  Then run `{{cappa}} agent check --json` once and make sure `pendingJudgments` is 0 and no result
56
- carries a `verdictError`.
59
+ carries a `verdictError`. (It exits 1 while anything differs — that is expected.)
57
60
 
58
61
  ## Reply
59
62
  Only a summary, one line per screenshot: `<name>: <label> — <headline>`, followed by any element
@@ -14,6 +14,8 @@ Current verdicts:
14
14
  (`intentional-likely`, `noise-likely`) means every changed screenshot with that verdict.
15
15
  3. Do not approve a `regression-likely` screenshot or a `deleted` one (approving a deletion removes
16
16
  its baseline) unless the request names it explicitly — if it does not, list it and ask.
17
+ `cappa agent approve` refuses `regression-likely` on its own; only when the user explicitly
18
+ confirmed that regression, add `--allow-regression`.
17
19
  4. Approve: `{{cappa}} agent approve <name> [<name> ...] --json`, or
18
20
  `{{cappa}} agent approve --label <label> --json`.
19
21
  5. Report what was approved in one line, and what was held back and why.
@@ -18,14 +18,17 @@ Current state of the last capture (`{{cappa}} agent check --json`):
18
18
  2. **Judge** — if `pendingJudgments > 0`, hand the pending screenshots to the `cappa-judge`
19
19
  subagent (Task tool) instead of reading the images yourself; it keeps the crops out of this
20
20
  conversation and records a verdict per screenshot. Then re-run `{{cappa}} agent check --json`.
21
- For a thorough second opinion on every change, run `{{cappa}} agent check --judge all --json`
22
- first so every changed screenshot gets a request.
21
+ For a thorough second opinion on every change — or when you doubt a heuristic label, e.g. a
22
+ `regression-likely` that matches a deliberate code change — run
23
+ `{{cappa}} agent check --judge all --json` first so every changed screenshot gets a request,
24
+ and have the subagent judge all of them (requests stay pending until judged).
23
25
  3. **Report** — one line per non-passing screenshot, grouped by label:
24
26
  `<name>: <label> — <headline>`, then `new` and `deleted` screenshots. End with
25
27
  `P/T passed · C changed · N new · D deleted`.
26
28
  4. **Offer next steps**, and wait for the user:
27
- - `regression-likely` → offer to find and fix the cause in the code, then re-capture just those
28
- tasks with `{{cappa}} capture --json --filter "<taskId>"` and review again.
29
+ - `regression-likely` → if the verdict is the heuristic's, check it against the code first.
30
+ Then offer to find and fix the cause, re-capture just those tasks with
31
+ `{{cappa}} capture --json --filter "<taskId>"`, and review again.
29
32
  - `intentional-likely` → ask to approve them: `/cappa-approve <names>`.
30
33
  - `noise-likely` → offer to mask the element (the skill's `MASKING.md`) or approve.
31
34
  - The verdicts are also visible in the review UI (`{{cappa}} review`).
@@ -1,7 +1,19 @@
1
1
  # cappa — judging changes
2
2
 
3
3
  Used when `cappa agent check --json` reports `pendingJudgments > 0`: the heuristic labelled those
4
- changes `ambiguous` and is deferring to you.
4
+ changes `ambiguous` and is deferring to you — or you asked for every change with `--judge all`.
5
+
6
+ ## Can you read images?
7
+ Judging is visual. If your model cannot read images, don't substitute your own tooling (no
8
+ ImageMagick, no manual crops, no pixel statistics). Hand the pending `judgment` directories to a
9
+ subagent whose model can read images — the `cappa-judge` subagent if `cappa agent init` installed
10
+ one — and have it follow this file; it records verdicts with `cappa agent judge` (or returns
11
+ labels and rationale for you to record).
12
+
13
+ ## Judge everything that is pending
14
+ A request stays pending through every later check until it has a verdict. After `--judge all`
15
+ that includes changes whose heuristic label you agree with — record those too (one call per
16
+ shared label), or `pendingJudgments` never reaches 0.
5
17
 
6
18
  ## Look for one shared cause first
7
19
  Before opening anything, group the pending `results[]` by `verdict.headline`. Headlines are built so
@@ -26,7 +38,8 @@ For each pending entry, in its `judgment` directory:
26
38
  continues outside its crop.
27
39
  5. You have the source code — use it. A diff that lines up with a change in the working tree or a
28
40
  recent commit is evidence of intent; one with no code change behind it is evidence of noise
29
- or a regression elsewhere.
41
+ or a regression elsewhere. The heuristic's label is one more signal, not the answer: it calls
42
+ structural changes `regression-likely` even when they were made on purpose.
30
43
 
31
44
  Labels:
32
45
  - `regression-likely` — broken, missing, clipped, overlapping, misaligned, wrong data, truncated
@@ -11,7 +11,8 @@ Cappa captures screenshots (Storybook stories, pages) with Playwright and diffs
11
11
  approved baselines. `cappa agent` turns the diff into something you can judge cheaply: a heuristic
12
12
  labels each change, and only the ones it cannot settle are handed to you with small crops.
13
13
 
14
- Run the CLI as `{{cappa}}` in this project; the examples below write `cappa` for short.
14
+ Run the CLI as `{{cappa}}` from the project root — it already runs in the package holding
15
+ `cappa.config.ts`, so don't `cd` first. The examples below write `cappa` for short.
15
16
 
16
17
  Sibling files — read on demand:
17
18
  - `JUDGING.md` — judging changes when `pendingJudgments > 0`.
@@ -28,8 +29,6 @@ Sibling files — read on demand:
28
29
  non-passing screenshot.
29
30
 
30
31
  ## 1. Capture
31
- Run from the directory holding `cappa.config.ts`.
32
-
33
32
  ```sh
34
33
  cappa capture --json
35
34
  ```
@@ -57,28 +56,48 @@ cappa agent check --json
57
56
  Returns `{ reportPath, total, passed, changed, new, deleted, pendingJudgments, results }`.
58
57
  `results` lists non-passing screenshots only, each
59
58
  `{ name, category, verdict?: { label, headline, action, source }, pending?, judgment? }`.
59
+ It exits 1 whenever anything is not passing — that is the normal result, not a failure. A real
60
+ failure prints `{ "error": "..." }` instead.
60
61
 
61
62
  - `category: "changed"` carries a `verdict`. `source: "heuristic"` came from the diff signals;
62
63
  `source: "agent"` is a verdict you (or another judge) wrote.
64
+ - A heuristic label is a guess from pixel signals, not a finding. In particular it calls any
65
+ medium- or high-severity structural change `regression-likely` — including content that was
66
+ added or removed on purpose. Before treating a heuristic `regression-likely` as broken, check it
67
+ against the code; if a deliberate change explains it, judge it (`--judge all`, below) instead of
68
+ "fixing" the feature.
63
69
  - `pending: true` → the heuristic said `ambiguous`; `judgment` is the directory with the evidence.
64
70
  **Read `JUDGING.md`**, judge them, then re-run `cappa agent check --json`.
65
- - `--judge all` requests a judgment for *every* changed screenshot, not only ambiguous ones — use it
66
- when the user wants a thorough review or you doubt a heuristic label.
71
+ - `--judge all` requests a judgment for *every* changed screenshot, not only ambiguous ones, so
72
+ each gets a `judgment` directory with `locator.png` and `regions.png`. Use it whenever you need
73
+ to look anyway: you doubt a heuristic label, or the user wants a thorough review. Then judge per
74
+ `JUDGING.md` like any other pending entry.
75
+ - Requests are sticky: once made, a request stays `pending` through every later check until it
76
+ gets a verdict, the screenshot is approved, or a re-capture changes its images. After
77
+ `--judge all`, judge every pending entry — including the ones whose heuristic label you agree
78
+ with (one `cappa agent judge` call per shared label is enough) — or `pendingJudgments` never
79
+ reaches 0.
80
+ - Judging means reading images. If you cannot read images yourself, don't improvise — hand the
81
+ `judgment` directories to a subagent whose model can (the `cappa-judge` subagent, when
82
+ `cappa agent init` installed one), let it judge per `JUDGING.md`, and use the verdicts it
83
+ records (or run `cappa agent judge` with the labels it returns).
67
84
 
68
85
  ## 3. Report and act
69
86
  One line per non-passing screenshot: `<name>: <label> — <headline>`. Then, per label:
70
87
 
71
- - `regression-likely` → something looks broken. Point the user at it. If they want it fixed (or
72
- you were asked to fix regressions), find the cause in the working tree (`git diff`, recent
73
- commits touching the component), fix the code, re-capture just that task with `--filter`, and
74
- re-check. Never approve a regression.
88
+ - `regression-likely` → something looks broken. If the verdict is the heuristic's
89
+ (`source: "heuristic"`), confirm it against the code first (see step 2). Point the user at it.
90
+ If they want it fixed (or you were asked to fix regressions), find the cause in the working
91
+ tree (`git diff`, recent commits touching the component), fix the code, re-capture just that
92
+ task with `--filter`, and re-check. Never approve a regression.
75
93
  - `intentional-likely` → a deliberate restyle or content change. Ask the user to confirm, then
76
94
  `cappa agent approve <name> [<name> ...] --json`.
77
95
  - `noise-likely` → anti-aliasing or non-deterministic content. Ask once: mask, approve or ignore.
78
96
  Prefer masking when the source is inherently unstable (clock, relative time, random avatar,
79
97
  animation) — approving only postpones the next failure. See `MASKING.md`.
80
- - `new` (no baseline yet) → the screenshot is new. Glance at it only if asked; approving makes it
81
- the baseline.
98
+ - `new` (no baseline yet) → the screenshot is new. There are no crops — looking means reading the
99
+ full-size image in `<outputDir>/actual/<name>.png`, so do it only if asked (and through an
100
+ image-capable subagent if you cannot read images). Approving makes it the baseline.
82
101
  - `deleted` (baseline without a capture) → the story/page was removed or failed to capture. Check
83
102
  capture output before approving: approving a deleted screenshot **deletes its baseline**.
84
103
 
@@ -87,9 +106,14 @@ screenshot whose current verdict has that label.
87
106
 
88
107
  ## Hard rules
89
108
  - Never approve without explicit user confirmation, and never approve `regression-likely`.
109
+ `cappa agent approve` refuses to; if the change is in fact intended, judge it
110
+ `intentional-likely` first. Pass `--allow-regression` only when the user explicitly tells you to
111
+ approve that regression.
90
112
  - `cappa agent approve` matches names exactly. Don't use plain `cappa approve` — its `--filter`
91
113
  is a substring match and approves far more than you intend.
92
114
  - Never edit files under `expected/`, `actual/`, `diff/` or `.cappa-agent/` by hand (except
93
115
  writing `verdict.json`).
116
+ - Never hand-roll crops, image diffs or pixel forensics (ImageMagick, scripts). Cappa's crops are
117
+ the evidence; if a screenshot has none, `cappa agent check --json --judge all` produces them.
94
118
  - In CI (`CI=true`), only `cappa agent check` is allowed; baselines change on developer machines.
95
119
  - Re-capturing invalidates verdicts for screenshots whose images changed — that is intended.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@cappa/cli",
3
- "version": "0.14.0",
3
+ "version": "0.15.1",
4
4
  "description": "Command line interface for Cappa",
5
5
  "author": "Tim Richter",
6
6
  "license": "MIT",