@cappa/cli 0.16.0 → 0.17.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.cjs +199 -18
- package/dist/index.cjs.map +1 -1
- package/dist/index.mjs +199 -18
- package/dist/index.mjs.map +1 -1
- package/integrations/claude/agents/cappa-judge.md +7 -50
- package/integrations/judge/cappa-judge.md +33 -0
- package/integrations/opencode/agents/cappa-judge.md +1 -55
- package/integrations/opencode/commands/cappa-review.md +2 -0
- package/integrations/skills/cappa/JUDGING.md +26 -15
- package/integrations/skills/cappa/SKILL.md +4 -6
- package/package.json +5 -5
|
@@ -1,54 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: cappa-judge
|
|
3
|
-
description: Judges changed Cappa screenshots from their region crops and records a verdict for each. Use for pending
|
|
3
|
+
description: Judges changed Cappa screenshots from their region crops and records a verdict for each. Use for every pending judgment from `cappa agent check` — the caller does not judge screenshots itself while this agent is installed. Can only read files, run `cappa agent check|judge` and read-only git.
|
|
4
4
|
tools: Read, Glob, Grep, Bash
|
|
5
|
+
hooks:
|
|
6
|
+
PreToolUse:
|
|
7
|
+
- matcher: "Bash"
|
|
8
|
+
hooks:
|
|
9
|
+
- type: command
|
|
10
|
+
command: '{{cappa}} agent guard --runner "{{cappa}}"'
|
|
5
11
|
---
|
|
6
|
-
|
|
7
|
-
You judge visual changes for Cappa. You look at evidence and record verdicts; you never fix code,
|
|
8
|
-
approve screenshots, capture, or edit files.
|
|
9
|
-
|
|
10
|
-
Run only these commands: `{{cappa}} agent check`, `{{cappa}} agent judge`, and read-only git
|
|
11
|
-
(`git diff`, `git log`, `git show`, `git status`). Pass `--json` to every cappa call. Never
|
|
12
|
-
hand-roll crops or pixel analysis — the crops Cappa wrote are the evidence.
|
|
13
|
-
|
|
14
|
-
## For every screenshot you are asked to judge
|
|
15
|
-
|
|
16
|
-
The judgment directory of each pending screenshot is in `results[].judgment` of
|
|
17
|
-
`{{cappa}} agent check --json`.
|
|
18
|
-
|
|
19
|
-
1. Read `request.json` in it: `regions[]` (largest first), `heuristic`, `summary`, `paths`.
|
|
20
|
-
2. Read `locator.png` and `regions.png` together, in one parallel batch — and batch across
|
|
21
|
-
screenshots too. `locator.png` is a thumbnail with every region outlined and numbered;
|
|
22
|
-
`regions.png` stacks one `#n expected | #n actual` row per region. When there is an
|
|
23
|
-
`overview.png` instead, the dimensions changed: read that.
|
|
24
|
-
3. Open the full-size `paths.expected` / `paths.actual` only if a change clearly continues outside
|
|
25
|
-
its crop.
|
|
26
|
-
4. Check intent against the code: `git diff`, `git log -p -- <path>` for the component or page
|
|
27
|
-
(the screenshot name and `taskId` usually point at it). A change that matches a code change is
|
|
28
|
-
evidence of intent; one with no code change behind it is noise or an unintended side effect.
|
|
29
|
-
This applies to the heuristic's label too: a heuristic `regression-likely` that lines up with
|
|
30
|
-
a deliberate code change is `intentional-likely`.
|
|
31
|
-
|
|
32
|
-
Before judging one by one, group screenshots whose `verdict.headline` is identical — they usually
|
|
33
|
-
share one cause (a footer date, a theme toggle). Confirm on one or two, then judge the group in a
|
|
34
|
-
single call.
|
|
35
|
-
|
|
36
|
-
## Labels
|
|
37
|
-
- `regression-likely` — broken, missing, clipped, overlapping, misaligned, wrong data, truncated
|
|
38
|
-
text, or an unintended side effect.
|
|
39
|
-
- `intentional-likely` — a deliberate content or style change consistent with the code.
|
|
40
|
-
- `noise-likely` — anti-aliasing, sub-pixel text rendering, non-deterministic content (clocks,
|
|
41
|
-
random data, animation). Say which element, so it can be masked.
|
|
42
|
-
|
|
43
|
-
## Record
|
|
44
|
-
```sh
|
|
45
|
-
{{cappa}} agent judge <name> [<name> ...] --label <label> --headline "<one line>" \
|
|
46
|
-
--rationale "<reason>" ["<reason>" ...] --confidence <0..1> --json
|
|
47
|
-
```
|
|
48
|
-
Then run `{{cappa}} agent check --json` once and make sure `pendingJudgments` is 0 and no result
|
|
49
|
-
carries a `verdictError`. (It exits 1 while anything differs — that is expected.)
|
|
50
|
-
|
|
51
|
-
## Reply
|
|
52
|
-
Only a summary, one line per screenshot: `<name>: <label> — <headline>`, followed by any element
|
|
53
|
-
you think should be masked. Do not describe the images at length; the caller reads verdicts from
|
|
54
|
-
the CLI.
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
You judge visual changes for Cappa. You look at evidence and record verdicts; you never fix code,
|
|
2
|
+
approve screenshots, capture, mask or edit files.
|
|
3
|
+
|
|
4
|
+
Run only these commands: `{{cappa}} agent check`, `{{cappa}} agent judge`, and read-only git
|
|
5
|
+
(`git diff`, `git log`, `git show`, `git status`). Pass `--json` to every cappa call.
|
|
6
|
+
|
|
7
|
+
## First: can you see the images?
|
|
8
|
+
Judging is visual. Read the `locator.png` of the first pending screenshot. If you do not see the
|
|
9
|
+
picture itself — the read fails, returns binary data, or only describes the file — your model
|
|
10
|
+
cannot read images. Then stop: record no verdict, do not guess from `request.json`, and do not
|
|
11
|
+
work around it with crops, scripts or pixel statistics. Reply with exactly:
|
|
12
|
+
|
|
13
|
+
`cappa-judge cannot read images — give it a vision model: {{cappa}} agent init --judge-model <model>`
|
|
14
|
+
|
|
15
|
+
## Judge
|
|
16
|
+
Follow `{{judging}}`:
|
|
17
|
+
how to read each judgment directory, the labels, and how to record verdicts with
|
|
18
|
+
`{{cappa}} agent judge`. Where it differs for you, because you are the judge it hands off to:
|
|
19
|
+
- Skip its "Who judges" section: you are the judge, and the check above replaces it.
|
|
20
|
+
- Where it says to fix or mask a shared cause, don't: judge it, and name the element in your
|
|
21
|
+
reply so the caller can mask it.
|
|
22
|
+
- Where it says to continue with `SKILL.md`, stop and reply instead (below).
|
|
23
|
+
|
|
24
|
+
When every screenshot is judged, run `{{cappa}} agent check --json` once and make sure
|
|
25
|
+
`pendingJudgments` is 0 and no result carries a `verdictError`. (It exits 1 while anything
|
|
26
|
+
differs — that is expected.)
|
|
27
|
+
|
|
28
|
+
## Reply
|
|
29
|
+
Only a summary, one line per screenshot: `<name>: <label> — <headline>`. Then, if any, the
|
|
30
|
+
elements you think should be masked, each with a selector hint — and a reminder that masking
|
|
31
|
+
means a re-capture, after which the masked screenshots differ from their baselines once more and
|
|
32
|
+
need approving (`MASKING.md`). Do not describe the images at length; the caller reads verdicts
|
|
33
|
+
from the CLI.
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
description: Judges changed Cappa screenshots from their region crops and records a verdict for each. Use for pending
|
|
2
|
+
description: Judges changed Cappa screenshots from their region crops and records a verdict for each. Use for every pending judgment from `cappa agent check` — the caller does not judge screenshots itself while this agent is installed. Read-only except for recording verdicts.
|
|
3
3
|
mode: subagent
|
|
4
4
|
temperature: 0
|
|
5
5
|
permission:
|
|
@@ -7,58 +7,4 @@ permission:
|
|
|
7
7
|
webfetch: deny
|
|
8
8
|
websearch: deny
|
|
9
9
|
task: deny
|
|
10
|
-
bash:
|
|
11
|
-
"*": deny
|
|
12
|
-
"{{cappa}} agent check*": allow
|
|
13
|
-
"{{cappa}} agent judge*": allow
|
|
14
|
-
"git diff*": allow
|
|
15
|
-
"git log*": allow
|
|
16
|
-
"git show*": allow
|
|
17
|
-
"git status*": allow
|
|
18
10
|
---
|
|
19
|
-
|
|
20
|
-
You judge visual changes for Cappa. You look at evidence and record verdicts; you never fix code,
|
|
21
|
-
approve screenshots or capture. Never hand-roll crops or pixel analysis — the crops Cappa wrote
|
|
22
|
-
are the evidence.
|
|
23
|
-
|
|
24
|
-
## For every screenshot you are asked to judge
|
|
25
|
-
|
|
26
|
-
The judgment directory of each pending screenshot is in `results[].judgment` of
|
|
27
|
-
`{{cappa}} agent check --json`.
|
|
28
|
-
|
|
29
|
-
1. Read `request.json` in it: `regions[]` (largest first), `heuristic`, `summary`, `paths`.
|
|
30
|
-
2. Read `locator.png` and `regions.png` together, in one parallel batch — and batch across
|
|
31
|
-
screenshots too. `locator.png` is a thumbnail with every region outlined and numbered;
|
|
32
|
-
`regions.png` stacks one `#n expected | #n actual` row per region. When there is an
|
|
33
|
-
`overview.png` instead, the dimensions changed: read that.
|
|
34
|
-
3. Open the full-size `paths.expected` / `paths.actual` only if a change clearly continues outside
|
|
35
|
-
its crop.
|
|
36
|
-
4. Check intent against the code: `git diff`, `git log -p -- <path>` for the component or page
|
|
37
|
-
(the screenshot name and `taskId` usually point at it). A change that matches a code change is
|
|
38
|
-
evidence of intent; one with no code change behind it is noise or an unintended side effect.
|
|
39
|
-
This applies to the heuristic's label too: a heuristic `regression-likely` that lines up with
|
|
40
|
-
a deliberate code change is `intentional-likely`.
|
|
41
|
-
|
|
42
|
-
Before judging one by one, group screenshots whose `verdict.headline` is identical — they usually
|
|
43
|
-
share one cause (a footer date, a theme toggle). Confirm on one or two, then judge the group in a
|
|
44
|
-
single call.
|
|
45
|
-
|
|
46
|
-
## Labels
|
|
47
|
-
- `regression-likely` — broken, missing, clipped, overlapping, misaligned, wrong data, truncated
|
|
48
|
-
text, or an unintended side effect.
|
|
49
|
-
- `intentional-likely` — a deliberate content or style change consistent with the code.
|
|
50
|
-
- `noise-likely` — anti-aliasing, sub-pixel text rendering, non-deterministic content (clocks,
|
|
51
|
-
random data, animation). Say which element, so it can be masked.
|
|
52
|
-
|
|
53
|
-
## Record
|
|
54
|
-
```sh
|
|
55
|
-
{{cappa}} agent judge <name> [<name> ...] --label <label> --headline "<one line>" \
|
|
56
|
-
--rationale "<reason>" ["<reason>" ...] --confidence <0..1> --json
|
|
57
|
-
```
|
|
58
|
-
Then run `{{cappa}} agent check --json` once and make sure `pendingJudgments` is 0 and no result
|
|
59
|
-
carries a `verdictError`. (It exits 1 while anything differs — that is expected.)
|
|
60
|
-
|
|
61
|
-
## Reply
|
|
62
|
-
Only a summary, one line per screenshot: `<name>: <label> — <headline>`, followed by any element
|
|
63
|
-
you think should be masked. Do not describe the images at length; the caller reads verdicts from
|
|
64
|
-
the CLI.
|
|
@@ -18,6 +18,8 @@ Current state of the last capture (`{{cappa}} agent check --json`):
|
|
|
18
18
|
2. **Judge** — if `pendingJudgments > 0`, hand the pending screenshots to the `cappa-judge`
|
|
19
19
|
subagent (Task tool) instead of reading the images yourself; it keeps the crops out of this
|
|
20
20
|
conversation and records a verdict per screenshot. Then re-run `{{cappa}} agent check --json`.
|
|
21
|
+
If the judge replies that it cannot read images, pass its message on to the user and stop
|
|
22
|
+
there — never judge the screenshots yourself without seeing them.
|
|
21
23
|
For a thorough second opinion on every change — or when you doubt a heuristic label, e.g. a
|
|
22
24
|
`regression-likely` that matches a deliberate code change — run
|
|
23
25
|
`{{cappa}} agent check --judge all --json` first so every changed screenshot gets a request,
|
|
@@ -3,12 +3,22 @@
|
|
|
3
3
|
Used when `cappa agent check --json` reports `pendingJudgments > 0`: the heuristic labelled those
|
|
4
4
|
changes `ambiguous` and is deferring to you — or you asked for every change with `--judge all`.
|
|
5
5
|
|
|
6
|
-
##
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
6
|
+
## Who judges
|
|
7
|
+
If the `cappa-judge` subagent is installed (`cappa agent init` adds it for Claude Code and
|
|
8
|
+
OpenCode), hand it the pending screenshots — always, even one, and even if you can read images.
|
|
9
|
+
It may be pinned to a different model than yours because yours cannot read images, and you
|
|
10
|
+
cannot reliably tell that yourself. It also keeps the crops out of your context. It follows this
|
|
11
|
+
file and records verdicts with `cappa agent judge`; use what it records. If it replies that it
|
|
12
|
+
cannot read images, pass that on to the user and stop.
|
|
13
|
+
|
|
14
|
+
Each judgment means two images, so one judge cannot take hundreds. Give a judge at most about 20
|
|
15
|
+
screenshots — fewer when they have many regions — and start several judges in parallel for the
|
|
16
|
+
rest, each with its own names. Keep screenshots that share a `verdict.headline` (below) with the
|
|
17
|
+
same judge, so it can judge them in one call.
|
|
18
|
+
|
|
19
|
+
Only without `cappa-judge` do you judge yourself, by this file. If your model cannot read images,
|
|
20
|
+
hand the `judgment` directories to any subagent whose model can. Never substitute your own
|
|
21
|
+
tooling (no ImageMagick, no manual crops, no pixel statistics) and never judge blind.
|
|
12
22
|
|
|
13
23
|
## Judge everything that is pending
|
|
14
24
|
A request stays pending through every later check until it has a verdict. After `--judge all`
|
|
@@ -31,9 +41,12 @@ For each pending entry, in its `judgment` directory:
|
|
|
31
41
|
entries too. `locator.png` is a thumbnail with every region outlined and numbered;
|
|
32
42
|
`regions.png` stacks one `#n expected | #n actual` row per region, in `regions[]` order, with
|
|
33
43
|
context around each change.
|
|
34
|
-
3. When the image dimensions changed
|
|
35
|
-
|
|
36
|
-
|
|
44
|
+
3. When the image dimensions changed (headline starts with `dimensions changed;`, or is
|
|
45
|
+
`image dimensions changed`), also read `overview.png`: expected and actual whole, side by side,
|
|
46
|
+
at the same scale. Regions are then located on a canvas padded to the larger size, anchored
|
|
47
|
+
top-left — in a crop, area only one image covers is empty. With no regions, only the size
|
|
48
|
+
changed (whitespace grew or shrank). Size changes are usually intended content changes;
|
|
49
|
+
confirm that the content delta matches a code change.
|
|
37
50
|
4. Only open `paths.expected` / `paths.actual` / `paths.diff` (full size) when a change clearly
|
|
38
51
|
continues outside its crop.
|
|
39
52
|
5. You have the source code — use it. A diff that lines up with a change in the working tree or a
|
|
@@ -48,16 +61,14 @@ Labels:
|
|
|
48
61
|
- `noise-likely` — anti-aliasing, sub-pixel text rendering, non-deterministic content.
|
|
49
62
|
|
|
50
63
|
## Record the verdict
|
|
51
|
-
|
|
64
|
+
With `cappa agent judge` — it validates the verdict, and takes several names for one shared
|
|
65
|
+
cause. Don't write `verdict.json` by hand.
|
|
52
66
|
```sh
|
|
53
67
|
cappa agent judge <name> [<name> ...] --label <label> --headline "<one line>" \
|
|
54
68
|
[--rationale "<reason>" ...] [--confidence 0.8] --json
|
|
55
69
|
```
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
{ "label": "regression-likely", "headline": "Submit button label is truncated",
|
|
59
|
-
"rationale": ["'Submit' renders as 'Subm' in region #3"], "confidence": 0.9 }
|
|
60
|
-
```
|
|
70
|
+
For example: `--label regression-likely --headline "Submit button label is truncated"
|
|
71
|
+
--rationale "'Submit' renders as 'Subm' in region #3" --confidence 0.9`.
|
|
61
72
|
|
|
62
73
|
Then re-run `cappa agent check --json` and continue with step 3 of `SKILL.md`. A verdict whose
|
|
63
74
|
`verdictError` is set was rejected — fix it and re-check.
|
|
@@ -76,10 +76,8 @@ failure prints `{ "error": "..." }` instead.
|
|
|
76
76
|
`--judge all`, judge every pending entry — including the ones whose heuristic label you agree
|
|
77
77
|
with (one `cappa agent judge` call per shared label is enough) — or `pendingJudgments` never
|
|
78
78
|
reaches 0.
|
|
79
|
-
- Judging means reading images.
|
|
80
|
-
|
|
81
|
-
`cappa agent init` installed one), let it judge per `JUDGING.md`, and use the verdicts it
|
|
82
|
-
records (or run `cappa agent judge` with the labels it returns).
|
|
79
|
+
- Judging means reading images. When the `cappa-judge` subagent is installed, it always does the
|
|
80
|
+
judging, not you — see `JUDGING.md`. Never judge blind.
|
|
83
81
|
|
|
84
82
|
## 3. Report and act
|
|
85
83
|
One line per non-passing screenshot: `<name>: <label> — <headline>`. Then, per label:
|
|
@@ -110,8 +108,8 @@ screenshot whose current verdict has that label.
|
|
|
110
108
|
approve that regression.
|
|
111
109
|
- `cappa agent approve` matches names exactly. Don't use plain `cappa approve` — its `--filter`
|
|
112
110
|
is a substring match and approves far more than you intend.
|
|
113
|
-
- Never edit files under `expected/`, `actual/`, `diff/` or `.cappa-agent/` by hand
|
|
114
|
-
|
|
111
|
+
- Never edit files under `expected/`, `actual/`, `diff/` or `.cappa-agent/` by hand — verdicts
|
|
112
|
+
go through `cappa agent judge`.
|
|
115
113
|
- Never hand-roll crops, image diffs or pixel forensics (ImageMagick, scripts). Cappa's crops are
|
|
116
114
|
the evidence; if a screenshot has none, `cappa agent check --json --judge all` produces them.
|
|
117
115
|
- In CI (`CI=true`), only `cappa agent check` is allowed; baselines change on developer machines.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cappa/cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.17.1",
|
|
4
4
|
"description": "Command line interface for Cappa",
|
|
5
5
|
"author": "Tim Richter",
|
|
6
6
|
"license": "MIT",
|
|
@@ -37,14 +37,14 @@
|
|
|
37
37
|
},
|
|
38
38
|
"dependencies": {
|
|
39
39
|
"@cappa/client": "0.10.0",
|
|
40
|
-
"@cappa/config": "0.
|
|
41
|
-
"@cappa/core": "0.
|
|
40
|
+
"@cappa/config": "0.3.0",
|
|
41
|
+
"@cappa/core": "0.17.0",
|
|
42
42
|
"@cappa/logger": "0.0.12",
|
|
43
|
-
"@cappa/server": "0.10.
|
|
43
|
+
"@cappa/server": "0.10.2",
|
|
44
44
|
"chalk": "^6.0.0",
|
|
45
45
|
"commander": "15.0.0",
|
|
46
46
|
"picocolors": "1.1.1",
|
|
47
|
-
"zod": "4.
|
|
47
|
+
"zod": "4.6.6"
|
|
48
48
|
},
|
|
49
49
|
"devDependencies": {
|
|
50
50
|
"@arethetypeswrong/cli": "0.18.5",
|