@petukhovart/agent-view 0.8.0 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +2 -2
- package/README.md +38 -76
- package/dist/cli/index.js +0 -0
- package/package.json +60 -62
- package/skills/verify/SKILL.md +22 -36
- package/skills/verify-recipe/SKILL.md +88 -147
- package/agents/design-conformance-runner.md +0 -107
- package/agents/verify-runner.md +0 -187
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-view",
|
|
3
|
-
"description": "Visual verification CLI for desktop apps (Electron/Tauri/Browser) via Chrome DevTools Protocol. Ships two skills (
|
|
4
|
-
"version": "0.8.
|
|
3
|
+
"description": "Visual verification CLI for desktop apps (Electron/Tauri/Browser) via Chrome DevTools Protocol. Ships two skills: verify (run DOM, screenshot, eval, watch, console checks against a live app) and verify-recipe (author a reusable verification plan as a .md file).",
|
|
4
|
+
"version": "0.8.1",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"cdp",
|
|
7
7
|
"electron",
|
package/README.md
CHANGED
|
@@ -29,8 +29,7 @@ Each step links to the full explanation below.
|
|
|
29
29
|
- **Worker access** — SharedWorker, ServiceWorker, dedicated Worker visible alongside pages; fuzzy `--target` resolution everywhere (id → title → URL)
|
|
30
30
|
- **Canvas / WebGL scene graph** — PixiJS today, engine-pluggable; `--compact` mirrors the DOM mode
|
|
31
31
|
- **Design-conformance verification** — pair screenshot commands with local design references (Figma export, hand-off
|
|
32
|
-
PNGs, any image on disk) inside a verify-recipe; the `
|
|
33
|
-
against the mockup
|
|
32
|
+
PNGs, any image on disk) inside a verify-recipe; the `verify` skill compares screenshots against the references inline
|
|
34
33
|
|
|
35
34
|
## Why CLI, not MCP?
|
|
36
35
|
|
|
@@ -57,8 +56,7 @@ npm install -g @petukhovart/agent-view
|
|
|
57
56
|
|
|
58
57
|
The plugin ships two skills. **`verify`** executes visual and runtime checks against a running app. **`verify-recipe`** generates a `.claude/verify-recipes/<slug>.md` file — a disciplined, cheapest-first command sequence for a feature or bugfix — that you or any AI agent can run later. Trigger it with phrases like "write a verify-recipe for the login fix" or "generate a verification plan for this feature".
|
|
59
58
|
|
|
60
|
-
For
|
|
61
|
-
see [Recommended workflow with Claude Code](#recommended-workflow-with-claude-code) below.
|
|
59
|
+
For the canonical author-once / re-run flow, see [Recommended workflow with Claude Code](#recommended-workflow-with-claude-code) below.
|
|
62
60
|
|
|
63
61
|
Verify:
|
|
64
62
|
|
|
@@ -194,13 +192,34 @@ Full surface in [Commands](#commands) below.
|
|
|
194
192
|
|
|
195
193
|
## Recommended workflow with Claude Code
|
|
196
194
|
|
|
197
|
-
A repeatable, token-efficient flow for "I shipped a feature/fix → confirm it actually works visually and at runtime".
|
|
198
|
-
Three phases, each driven by a focused prompt.
|
|
195
|
+
A repeatable, token-efficient flow for "I shipped a feature/fix → confirm it actually works visually and at runtime". Two phases, each driven by a focused prompt.
|
|
199
196
|
|
|
200
|
-
|
|
197
|
+
```mermaid
|
|
198
|
+
flowchart TD
|
|
199
|
+
Dev["Developer<br/>in Claude Code"] -->|prompt| Agent["Main Claude agent<br/>(Opus / Sonnet)"]
|
|
201
200
|
|
|
202
|
-
|
|
203
|
-
|
|
201
|
+
Agent -->|"verify ad-hoc"| Verify["verify skill"]
|
|
202
|
+
Agent -->|"write recipe"| Recipe["verify-recipe skill"]
|
|
203
|
+
|
|
204
|
+
Recipe -->|interview| Dev
|
|
205
|
+
Recipe -->|writes| File[".claude/verify-recipes/<slug>.md"]
|
|
206
|
+
File -.->|read on next run| Verify
|
|
207
|
+
|
|
208
|
+
Verify -->|"agent-view dom / eval / click /<br/>screenshot / watch / console"| CLI["agent-view CLI"]
|
|
209
|
+
CLI -->|CDP| App["Live app<br/>(Electron / Tauri / Browser)"]
|
|
210
|
+
App -->|stdout / image paths| Verify
|
|
211
|
+
|
|
212
|
+
Verify -->|"pass / fail summary<br/>+ design conformance verdict"| Dev
|
|
213
|
+
|
|
214
|
+
classDef skill fill:#e8f0ff,stroke:#3060a0,color:#0a1f3d
|
|
215
|
+
classDef tool fill:#fff4d6,stroke:#a07020,color:#3d2a05
|
|
216
|
+
class Verify,Recipe skill
|
|
217
|
+
class CLI tool
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
### Phase 1 — Author the verification plan (once)
|
|
221
|
+
|
|
222
|
+
Generate the recipe **once**, from a PRD / plan file / Jira ticket / commit range. The recipe is reusable — re-run after every iteration on the same feature.
|
|
204
223
|
|
|
205
224
|
```text
|
|
206
225
|
Generate a verify-recipe for the changes in commits <hash1>..<hash2>.
|
|
@@ -212,41 +231,24 @@ Design references (if any):
|
|
|
212
231
|
- /abs/path/figma-exports/error-state.png → label "invalid creds error"
|
|
213
232
|
```
|
|
214
233
|
|
|
215
|
-
What this triggers: the `verify-recipe` skill interviews you (if more context needed), then writes a
|
|
216
|
-
`.claude/verify-recipes/<slug>.md` with `Repro Steps`, `Evidence Commands` (cheapest-first: `eval` / `dom --filter`
|
|
217
|
-
before `screenshot`), `Regression Checks`, and — if you provided design refs — a `Design Conformance` table mapping
|
|
218
|
-
screenshot commands to expected reference images.
|
|
234
|
+
What this triggers: the `verify-recipe` skill interviews you (if more context needed), then writes a `.claude/verify-recipes/<slug>.md` with `Repro Steps`, `Evidence Commands` (cheapest-first: `eval` / `dom --filter` before `screenshot`), `Regression Checks`, and — if you provided design refs — a `Design Conformance` table mapping screenshot commands to expected reference images.
|
|
219
235
|
|
|
220
|
-
**Tip:** if you implemented from a Figma file via the `figma-implement-design` skill, it likely already saved exports
|
|
221
|
-
somewhere on disk — pass those paths. agent-view does NOT fetch from Figma URLs; provide local files only.
|
|
236
|
+
**Tip:** if you implemented from a Figma file via the `figma-implement-design` skill, it likely already saved exports somewhere on disk — pass those paths. agent-view does NOT fetch from Figma URLs; provide local files only.
|
|
222
237
|
|
|
223
|
-
### Phase 2 — Run the recipe
|
|
238
|
+
### Phase 2 — Run the recipe
|
|
224
239
|
|
|
225
240
|
```text
|
|
226
241
|
Run the verify-recipe at .claude/verify-recipes/<slug>.md.
|
|
227
242
|
```
|
|
228
243
|
|
|
229
|
-
What this triggers: the `verify` skill
|
|
230
|
-
subagent (Haiku) to execute every step and return a compact JSON report. If the recipe has a `Design Conformance`
|
|
231
|
-
section, `design-conformance-runner` (also Haiku) runs in parallel and visually compares screenshots against the
|
|
232
|
-
references.
|
|
233
|
-
|
|
234
|
-
**Effect on cost:** raw output (DOM dumps, screenshots, eval values) stays in the subagent. The main agent only sees the
|
|
235
|
-
merged JSON report — typically <2k tokens vs ~50k if Opus ran the recipe directly.
|
|
244
|
+
What this triggers: the `verify` skill reads the recipe, performs the `Repro Steps` setup, runs each `Evidence Command` against the live app, compares output to the recipe's `Expected:` lines, and reports pass/fail per step. If the recipe has a `Design Conformance` section, the same skill captures the screenshots, opens both actual and expected images, and reports `match` / `minor_mismatch` / `major_mismatch` per pair — all inline, in the same conversation.
|
|
236
245
|
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
The merged report lists `failed`, `requires_visual_review`, and `major_mismatch` items with one-sentence diagnoses. Hand
|
|
240
|
-
any subset back to the main agent:
|
|
246
|
+
When something fails, ask the main agent to fix and re-run only the affected steps:
|
|
241
247
|
|
|
242
248
|
```text
|
|
243
|
-
|
|
244
|
-
and step 7 (selective filter heading missing). Re-run only those two steps after the fix.
|
|
249
|
+
Step 4 failed (zone filter not mutating store). Fix and re-run that step plus step 7.
|
|
245
250
|
```
|
|
246
251
|
|
|
247
|
-
The main agent re-runs only the failing commands itself for richer evidence, makes the fix, and re-spawns
|
|
248
|
-
`verify-runner` for the targeted re-check — not the whole recipe.
|
|
249
|
-
|
|
250
252
|
### One-shot prompt (when there's no plan to convert)
|
|
251
253
|
|
|
252
254
|
For small fixes where you don't want a persistent recipe file:
|
|
@@ -259,53 +261,13 @@ I added a `saving` ref and bound it to :disabled. Verify it works:
|
|
|
259
261
|
- visual: button greys out (compare to /abs/path/saving-state.png if one is provided)
|
|
260
262
|
```
|
|
261
263
|
|
|
262
|
-
Claude will pick the right skill (usually `verify` ad-hoc mode), run a handful of `eval` / `dom --filter` / `console`
|
|
263
|
-
calls, and only screenshot if the visual claim needs it.
|
|
264
|
-
|
|
265
|
-
### Three-phase preconditions (0.8.0+)
|
|
266
|
-
|
|
267
|
-
Recipes split setup into three phases, each with a different runner:
|
|
268
|
-
|
|
269
|
-
| Phase | Who runs it | Purpose | Idempotent? |
|
|
270
|
-
|---|---|---|---|
|
|
271
|
-
| `## Manual Preconditions` | Human (or main agent) | Only what `agent-view` physically can't (USB token, hardware, multi-machine state). Usually empty. | n/a |
|
|
272
|
-
| `## Bringup` | verify-runner | Conditional + idempotent setup: `if <state-check> is falsy → run actions → wait for state to settle`. Logins, mounting widgets, navigation, anything `agent-view` can drive deterministically. | YES per step |
|
|
273
|
-
| `## Machine Preconditions` | verify-runner | Pure state queries that confirm Bringup actually landed. No actions here. | trivially |
|
|
274
|
-
|
|
275
|
-
Bringup steps look like:
|
|
276
|
-
|
|
277
|
-
```markdown
|
|
278
|
-
### B1. Login if on auth screen
|
|
279
|
-
- if `agent-view eval "typeof window.__dev"` is not `"object"`:
|
|
280
|
-
agent-view fill --filter "Логин" "root"
|
|
281
|
-
agent-view fill --filter "Пароль" "$AGENTVIEW_PASSWORD"
|
|
282
|
-
agent-view click --filter "Войти"
|
|
283
|
-
wait for `agent-view eval "typeof window.__dev"` to be `"object"`, timeout 15s
|
|
284
|
-
```
|
|
285
|
-
|
|
286
|
-
The runner always evaluates the IF first (cheap). If the system is already in the target state → skip the actions, advance. Otherwise → run actions in order, then poll the post-condition until truthy or timeout. **Re-running a recipe when the app is ready costs ~3 seconds** (only state-checks, zero actions). Starting from auth screen costs ~15-25 seconds (one-time login + mount + navigate). Same recipe, both starting states, no manual intervention.
|
|
287
|
-
|
|
288
|
-
Failure modes are distinct:
|
|
289
|
-
- `bringup_failed` → bringup spec is wrong (the action commands ran but the post-condition didn't land — likely the programmatic API doesn't exist or the button text changed). Author error.
|
|
290
|
-
- `precondition_failed` → bringup ran AND post-conditions landed, but Machine Preconditions still false. Real environment issue (degraded mode, wrong user role, missing data).
|
|
291
|
-
- `cascading_failures` → bringup + preconditions OK, but Evidence Commands fail. Verified feature is broken or recipe is stale.
|
|
292
|
-
|
|
293
|
-
Without the three-phase split, all three look like "step N failed" and you can't tell them apart.
|
|
294
|
-
|
|
295
|
-
**Credentials in Bringup:** always use env-var references (`$AGENTVIEW_PASSWORD`), never inline a literal password. Recipes get committed to git. The `verify-recipe` skill warns when it sees a literal-looking password and offers to convert.
|
|
264
|
+
Claude will pick the right skill (usually `verify` ad-hoc mode), run a handful of `eval` / `dom --filter` / `console` calls, and only screenshot if the visual claim needs it.
|
|
296
265
|
|
|
297
266
|
### Anti-patterns to avoid
|
|
298
267
|
|
|
299
|
-
- "Just verify the feature" with no plan or symptom — the recipe author can't pick the cheapest signal without knowing
|
|
300
|
-
|
|
301
|
-
-
|
|
302
|
-
first.
|
|
303
|
-
- Running the recipe directly with Opus when a recipe file exists — that's exactly the case `verify-runner` is for.
|
|
304
|
-
Prefer Phase 2's prompt.
|
|
305
|
-
- Stuffing 50 assertions into one recipe — split per-feature. A recipe should run in <2 minutes and produce a report you
|
|
306
|
-
can read in 30 seconds.
|
|
307
|
-
- Manual Preconditions without Machine Precondition counterparts — the runner can't catch the user skipping setup. The
|
|
308
|
-
skill warns during authoring; either add a check or accept the gap explicitly in the recipe.
|
|
268
|
+
- "Just verify the feature" with no plan or symptom — the recipe author can't pick the cheapest signal without knowing what "works" means. Give it the symptom that motivated the fix.
|
|
269
|
+
- Pasting Figma URLs and expecting agent-view to download them — it won't. Export the frames you care about to PNG first.
|
|
270
|
+
- Stuffing 50 assertions into one recipe — split per-feature. A recipe should run in <2 minutes and produce a report you can read in 30 seconds.
|
|
309
271
|
|
|
310
272
|
## How it works
|
|
311
273
|
|
package/dist/cli/index.js
CHANGED
|
File without changes
|
package/package.json
CHANGED
|
@@ -1,62 +1,60 @@
|
|
|
1
|
-
{
|
|
2
|
-
"name": "@petukhovart/agent-view",
|
|
3
|
-
"version": "0.8.
|
|
4
|
-
"description": "CLI tool for visual verification of desktop apps (Electron/Tauri) via Chrome DevTools Protocol",
|
|
5
|
-
"type": "module",
|
|
6
|
-
"main": "dist/index.js",
|
|
7
|
-
"bin": {
|
|
8
|
-
"agent-view": "dist/cli/index.js"
|
|
9
|
-
},
|
|
10
|
-
"files": [
|
|
11
|
-
"dist",
|
|
12
|
-
".claude-plugin",
|
|
13
|
-
"skills",
|
|
14
|
-
"
|
|
15
|
-
"
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
"
|
|
20
|
-
"
|
|
21
|
-
"
|
|
22
|
-
"
|
|
23
|
-
"
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
"
|
|
27
|
-
"
|
|
28
|
-
"
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
"
|
|
34
|
-
"
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
"
|
|
40
|
-
"
|
|
41
|
-
"
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
"
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
"
|
|
49
|
-
"node": "
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
"
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
"
|
|
58
|
-
"
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
}
|
|
62
|
-
}
|
|
1
|
+
{
|
|
2
|
+
"name": "@petukhovart/agent-view",
|
|
3
|
+
"version": "0.8.1",
|
|
4
|
+
"description": "CLI tool for visual verification of desktop apps (Electron/Tauri) via Chrome DevTools Protocol",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"main": "dist/index.js",
|
|
7
|
+
"bin": {
|
|
8
|
+
"agent-view": "dist/cli/index.js"
|
|
9
|
+
},
|
|
10
|
+
"files": [
|
|
11
|
+
"dist",
|
|
12
|
+
".claude-plugin",
|
|
13
|
+
"skills",
|
|
14
|
+
"README.md",
|
|
15
|
+
"LICENSE"
|
|
16
|
+
],
|
|
17
|
+
"keywords": [
|
|
18
|
+
"cdp",
|
|
19
|
+
"chrome-devtools-protocol",
|
|
20
|
+
"electron",
|
|
21
|
+
"tauri",
|
|
22
|
+
"desktop",
|
|
23
|
+
"testing",
|
|
24
|
+
"accessibility",
|
|
25
|
+
"screenshot",
|
|
26
|
+
"automation",
|
|
27
|
+
"ai-agent",
|
|
28
|
+
"cli"
|
|
29
|
+
],
|
|
30
|
+
"author": "Artem Petukhov",
|
|
31
|
+
"license": "MIT",
|
|
32
|
+
"repository": {
|
|
33
|
+
"type": "git",
|
|
34
|
+
"url": "git+https://github.com/PetukhovArt/agent-view.git"
|
|
35
|
+
},
|
|
36
|
+
"bugs": {
|
|
37
|
+
"url": "https://github.com/PetukhovArt/agent-view/issues"
|
|
38
|
+
},
|
|
39
|
+
"homepage": "https://github.com/PetukhovArt/agent-view#readme",
|
|
40
|
+
"engines": {
|
|
41
|
+
"node": ">=20"
|
|
42
|
+
},
|
|
43
|
+
"dependencies": {
|
|
44
|
+
"chrome-remote-interface": "^0.34.0",
|
|
45
|
+
"commander": "^14.0.3",
|
|
46
|
+
"fast-json-patch": "^3.1.1"
|
|
47
|
+
},
|
|
48
|
+
"devDependencies": {
|
|
49
|
+
"@types/node": "^25.5.0",
|
|
50
|
+
"tsx": "^4.21.0",
|
|
51
|
+
"typescript": "^6.0.2",
|
|
52
|
+
"vitest": "^4.1.2"
|
|
53
|
+
},
|
|
54
|
+
"scripts": {
|
|
55
|
+
"build": "tsc",
|
|
56
|
+
"dev": "tsx src/cli/index.ts",
|
|
57
|
+
"test": "vitest run",
|
|
58
|
+
"test:watch": "vitest"
|
|
59
|
+
}
|
|
60
|
+
}
|
package/skills/verify/SKILL.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: verify
|
|
3
3
|
description: "Visual + runtime verification of desktop apps via Chrome DevTools Protocol. Use when modifying UI components, fixing visual bugs, testing user interactions, verifying layout, or when any workflow phase needs to inspect the running application — DOM, screenshots, scene graph, runtime state in pages and SharedWorkers/ServiceWorkers, console errors, or reactive-state diffs over time. Triggers on: verify, check UI, test how it looks, visual regression, screenshot, inspect DOM, check store/state, watch state changes, what changed after click, wait until state, read worker, console errors, runtime check, eval in page."
|
|
4
|
-
allowed-tools: Bash(agent-view *)
|
|
4
|
+
allowed-tools: Bash(agent-view *), Read
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
# Visual Verification with agent-view
|
|
@@ -178,44 +178,18 @@ When two tools could answer the same question, prefer the one higher up the tabl
|
|
|
178
178
|
|
|
179
179
|
## Verification Workflow
|
|
180
180
|
|
|
181
|
-
### Recipe Execution Mode (
|
|
181
|
+
### Recipe Execution Mode (when a recipe file exists)
|
|
182
182
|
|
|
183
|
-
If the developer points you at a `.claude/verify-recipes/<slug>.md` file,
|
|
184
|
-
|
|
185
|
-
Why: recipe execution is mechanical (run commands, compare to `Expected:`, report). It does not need Opus-level reasoning, but the raw output (DOM dumps, screenshots, eval results) easily exceeds 30k tokens of context noise. Delegating to a Haiku subagent keeps your context clean and cuts cost ~10×.
|
|
186
|
-
|
|
187
|
-
**Pre-flight check (do this BEFORE spawning the runner):**
|
|
183
|
+
If the developer points you at a `.claude/verify-recipes/<slug>.md` file, or one is discoverable via `ls .claude/verify-recipes/`, execute it inline yourself — **no subagent**.
|
|
188
184
|
|
|
189
185
|
1. `Read` the recipe.
|
|
190
|
-
2.
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
4. Resolve the window id once: `agent-view discover` → pick the main window's id.
|
|
198
|
-
|
|
199
|
-
**Spawn:**
|
|
200
|
-
|
|
201
|
-
5. Spawn `verify-runner` via the Agent tool with a prompt containing:
|
|
202
|
-
- Absolute `recipe_path`
|
|
203
|
-
- Resolved `window_id`
|
|
204
|
-
- `mode: full` (or `dry_run` if the user asked to validate the recipe first)
|
|
205
|
-
- Any extra context the user provided
|
|
206
|
-
6. Wait for the JSON report.
|
|
207
|
-
|
|
208
|
-
**Handle the report:**
|
|
209
|
-
|
|
210
|
-
7. **If `status: bringup_failed`** → the bringup spec is wrong: an action command failed to land its post-condition (e.g., `selectLocationAndFly` doesn't exist, login button text changed, programmatic API renamed). Surface `failed_bringup_step` and the post-condition's actual value. Suggest re-authoring the broken Bringup step via `verify-recipe`. Do NOT click around manually trying to make it work.
|
|
211
|
-
8. **If `status: bringup_budget_exhausted` or `bringup_timeout`** → bringup is taking too long. Either an animation is slower than the recipe expects (extend `timeout` on the offending step), or bringup is fighting an unexpected app state. Show the bringup transcript and ask the user.
|
|
212
|
-
9. **If `status: precondition_failed`** → bringup completed but state still wrong. Could be: (a) bringup is incomplete (missing a step), (b) a Machine Precondition is overly strict, (c) real environment issue. Relay both the `bringup` block (showing what bringup did) and the `failed_precondition` so the user sees the full picture.
|
|
213
|
-
10. **If `status: cascading_failures` or `budget_exhausted`** → the recipe Evidence is likely stale (selectors/refs changed, UI restructured). Show the first 1-2 failed Evidence steps and ask whether to update the recipe via `verify-recipe` or investigate manually.
|
|
214
|
-
11. **If `design_conformance_section: true`** → also spawn `design-conformance-runner` in parallel with the `design_conformance_pairs` array. Merge both reports.
|
|
215
|
-
12. **If `requires_visual_review` steps** have no design ref attached → open each screenshot yourself with `Read` and decide pass/fail.
|
|
216
|
-
13. **If individual `failed` steps** in an otherwise-completed run → re-run that specific failing command yourself for richer evidence. Do not re-execute the whole recipe.
|
|
217
|
-
|
|
218
|
-
Output to the user: a tight summary — what passed, what failed, what needs visual review, and (if any) which design conformance issues to fix. Do not paste the raw JSON unless asked.
|
|
186
|
+
2. If it has a `## Repro Steps` section, follow them — log in, navigate, set up data — using `agent-view` commands. Modern recipes (0.6+) may instead have `## Manual Preconditions` / `## Bringup` / `## Machine Preconditions` sections; treat those as documentation describing the expected state and execute the actions inline. There is no formal DSL — read what the section says and do it.
|
|
187
|
+
3. Resolve the window id once with `agent-view discover` if you need `--window`.
|
|
188
|
+
4. Run each `## Evidence Commands` subsection in order. Compare output to its `Expected:` line. Mark each as `pass` / `fail` / `requires_visual_review`.
|
|
189
|
+
5. After 2–3 consecutive failures, stop and flag the recipe as likely stale rather than continuing to burn tool calls.
|
|
190
|
+
6. Run `## Regression Checks` last.
|
|
191
|
+
7. If a `## Design Conformance` section is present, run the inline workflow below.
|
|
192
|
+
8. Report a tight summary to the user: passed / failed / visual-review counts plus one-liner per failure. Don't paste raw stdout unless asked.
|
|
219
193
|
|
|
220
194
|
### Ad-hoc Mode (standalone)
|
|
221
195
|
|
|
@@ -242,6 +216,18 @@ When UI scenarios are pre-generated (e.g., from a plan file with `## UI Scenario
|
|
|
242
216
|
|
|
243
217
|
This mode works with any workflow that generates plan files with UI scenarios.
|
|
244
218
|
|
|
219
|
+
### Design Conformance (inline)
|
|
220
|
+
|
|
221
|
+
When a recipe contains a `## Design Conformance` table — `(label, screenshot command, expected reference path)` rows — execute it yourself, no subagent.
|
|
222
|
+
|
|
223
|
+
For each row:
|
|
224
|
+
1. Run the screenshot command (capture the saved file path from stdout).
|
|
225
|
+
2. `Read` both the captured image and the `expected_path`. If `expected_path` doesn't exist or is unreadable, mark the pair `skipped (expected_missing)` and move on.
|
|
226
|
+
3. Compare visually for: layout (relative position, alignment), sizing, color (dominant color family), typography (weight/size broadly), content presence (anything missing or extra), decorations (borders, shadows, dashed/solid lines, icons).
|
|
227
|
+
4. Report each pair as `match` / `minor_mismatch` / `major_mismatch` with a one-sentence deviation. Major = missing/wrong component, broken layout, wrong color family, wrong text content. Minor = <10px spacing drift, slight color shade, small decoration difference.
|
|
228
|
+
|
|
229
|
+
Tolerance default: a designer's code-review level — flag what they'd notice, ignore anti-aliasing noise. Don't speculate about CSS causes — describe what looks different and let the parent / user decide.
|
|
230
|
+
|
|
245
231
|
## Resilience
|
|
246
232
|
|
|
247
233
|
- **Stale refs:** After HMR, navigation, or state change — re-run `dom` for fresh refs before interacting
|
|
@@ -10,43 +10,26 @@ You help the developer author a disciplined, cheapest-first verification recipe
|
|
|
10
10
|
|
|
11
11
|
## What this produces
|
|
12
12
|
|
|
13
|
-
A file at `.claude/verify-recipes/<kebab-slug>.md`
|
|
13
|
+
A file at `.claude/verify-recipes/<kebab-slug>.md` containing:
|
|
14
14
|
|
|
15
|
-
- **
|
|
16
|
-
- **
|
|
17
|
-
- **
|
|
18
|
-
- **
|
|
19
|
-
- **
|
|
15
|
+
- **REPRO STEPS** — exact state the app must be in before checks run (free-form prose)
|
|
16
|
+
- **NARROWED SIGNAL** — the measurable indicator that proves success or failure
|
|
17
|
+
- **EVIDENCE COMMANDS** — ordered `agent-view` calls, cheapest first, each annotated with what it proves
|
|
18
|
+
- **POSITIVE-CASE ASSERTIONS** — what "pass" looks like for each command
|
|
19
|
+
- **REGRESSION CHECKS** — adjacent paths that must not have broken
|
|
20
|
+
- **DESIGN CONFORMANCE** *(optional)* — pairs of `(label, screenshot command, expected reference image)` to compare against local design exports. Executed inline by the `verify` skill — no subagent.
|
|
20
21
|
|
|
21
22
|
Create the directory if missing: `mkdir -p .claude/verify-recipes`
|
|
22
23
|
|
|
23
|
-
## Why three precondition phases
|
|
24
|
-
|
|
25
|
-
| Phase | Who runs it | What it does | Idempotent? |
|
|
26
|
-
|---|---|---|---|
|
|
27
|
-
| `## Manual Preconditions` | Human (or main agent) | Only what `agent-view` physically can't (USB token, hardware setup, multi-machine state) | n/a |
|
|
28
|
-
| `## Bringup` | verify-runner | `if state-check is falsy → action → wait`. Logins, mounting widgets, navigation — anything `agent-view` can do deterministically | YES (per step) |
|
|
29
|
-
| `## Machine Preconditions` | verify-runner | Pure `eval` / `dom --filter` state queries. NO actions. | trivially |
|
|
30
|
-
|
|
31
|
-
**Why split Bringup from Machine Preconditions?** A failure tells you a different thing in each:
|
|
32
|
-
- A failed Bringup post-condition → bringup spec is wrong (e.g., `selectLocationAndFly` doesn't exist; "Войти" button text changed). Author error.
|
|
33
|
-
- A failed Machine Precondition after Bringup succeeded → bringup THINKS it set up the state but the state isn't there (e.g., login appeared to succeed but the user role is wrong, the app is in a degraded mode). Real environment issue.
|
|
34
|
-
- A failed Evidence Command after both passed → real bug in the verified feature.
|
|
35
|
-
|
|
36
|
-
Without the split, all three look like "step N failed" and the developer can't tell which.
|
|
37
|
-
|
|
38
|
-
**Why keep Manual at all?** Some things really can't be automated cheaply: physical USB-key auth, multi-machine setups, long-running data migrations. List them — but be honest, this section should usually be empty.
|
|
39
|
-
|
|
40
24
|
## Methodology
|
|
41
25
|
|
|
42
|
-
Frame the recipe with **hard-debug** discipline
|
|
26
|
+
Frame the recipe with **hard-debug** discipline: REPRO → narrowed signal → minimize scope → root-cause check → fix verification.
|
|
43
27
|
|
|
44
|
-
1. Start from
|
|
45
|
-
2. Convert vague expectations ("looks right") into measurable signals (
|
|
28
|
+
1. Start from a reproducible starting state, not "open the app and poke around".
|
|
29
|
+
2. Convert vague expectations ("looks right") into measurable signals (`store.user.role === 'admin'`).
|
|
46
30
|
3. Prefer the cheapest tool that can answer the question — a value check costs ~50 tokens, a screenshot costs ~6 000.
|
|
47
31
|
4. Include at least one negative-case check (the old symptom must no longer appear).
|
|
48
32
|
5. Include at least one regression check (an adjacent flow must still work).
|
|
49
|
-
6. **Bringup steps must be conditional + idempotent.** Every action wrapped in `if <state-check> is falsy`. Every action followed by `wait for <post-condition> to be truthy`. Re-running the recipe must be a no-op when the app is already ready.
|
|
50
33
|
|
|
51
34
|
## Tool-cost decision tree
|
|
52
35
|
|
|
@@ -63,50 +46,28 @@ Pick the first row that can answer the question. Only go lower when the row abov
|
|
|
63
46
|
| Canvas / WebGL scene state | `agent-view scene --diff` | DOM is empty for canvas apps |
|
|
64
47
|
|
|
65
48
|
**Anti-patterns to reject:**
|
|
66
|
-
-
|
|
67
|
-
-
|
|
68
|
-
-
|
|
69
|
-
-
|
|
70
|
-
- Hardcoded credentials in Bringup commands. Use env-var references (`"$AGENTVIEW_PASSWORD"`) — never inline a password, even for dev/masterkey accounts. Recipes get committed to git.
|
|
71
|
-
- Opening Evidence Commands with a screenshot to "see the state" — use `dom --filter` or `eval` first.
|
|
72
|
-
- "Check that it looks right" — every Evidence assertion must be a concrete pass/fail criterion. The single legitimate exception is the `## Design Conformance` section.
|
|
49
|
+
- Opening with a screenshot to "see the state" — use `dom --filter` or `eval` first.
|
|
50
|
+
- Using `eval` when `dom --filter` answers the question.
|
|
51
|
+
- Assertions that depend on transient state without `watch --until` to stabilize first.
|
|
52
|
+
- "Check that it looks right" — every assertion must be a concrete pass/fail criterion. The single legitimate exception is the optional `## Design Conformance` section.
|
|
73
53
|
- Inventing design reference paths (`.figma-refs/...`) when the developer did not provide them. No refs → no Design Conformance section.
|
|
74
54
|
|
|
75
55
|
## Workflow
|
|
76
56
|
|
|
77
|
-
### Step 1 — gather context
|
|
57
|
+
### Step 1 — gather context
|
|
78
58
|
|
|
79
|
-
When invoked, ask in plain text
|
|
80
|
-
|
|
81
|
-
**Block A — what's being verified:**
|
|
59
|
+
When invoked, ask the developer in plain text (no tool calls yet):
|
|
82
60
|
|
|
83
61
|
1. What was shipped or fixed? (feature name or bug description)
|
|
84
62
|
2. What was the original symptom or expected behavior?
|
|
85
63
|
3. Any known failure mode or edge case to cover?
|
|
64
|
+
4. *(Optional)* Any local design reference images to compare screenshots against (Figma exports, hand-off PNGs, screenshots from disk)? **Local files only** — agent-view does NOT fetch from Figma URLs. If none — skip the Design Conformance section.
|
|
86
65
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
4. **Authentication** — does the app require login before the feature is reachable? If yes:
|
|
90
|
-
- What identifies "logged out" vs "logged in" in JS state? (e.g., `typeof window.__dev`, `!!store.user`)
|
|
91
|
-
- Login flow: which fields/buttons (`fill --filter "..."`, `click --filter "..."`) and what credentials? **Use an env-var, never inline.** Suggest a name like `AGENTVIEW_PASSWORD` and tell the user to `export` it in their shell or `.env.local`.
|
|
92
|
-
5. **UI mode / view requirements** — does the feature live behind a specific app mode/view that must be active before it appears (e.g., "map view, not settings panel"; "edit mode, not view mode"; "modal X must be open")? For each:
|
|
93
|
-
- State check that proves this mode is active (e.g., `document.querySelector('.cesium-widget') !== null` for map mode).
|
|
94
|
-
- Action(s) to enter this mode if not active. Prefer programmatic API (e.g., `eval "store.setMode('map')"`) over UI clicks. If only UI works, list the click sequence.
|
|
95
|
-
6. **Setup actions** — beyond modes, what other setup must happen (drag widget into cell, navigate camera to a location, open a dialog)? For each:
|
|
96
|
-
- Programmatic API if it exists (e.g., `gis-widget-root.selectLocationAndFly(uuid)`, `workspace.addWidget(...)`). Prefer this — much more reliable than UI automation.
|
|
97
|
-
- State check that proves the setup happened.
|
|
98
|
-
- Wait timing: how long does this take to settle? (camera animation, async data load).
|
|
99
|
-
|
|
100
|
-
If the developer doesn't know whether a programmatic API exists, ask them to point you at the relevant store/composable/component file — you can read it and find one (or confirm none).
|
|
101
|
-
|
|
102
|
-
**Block C — verification body and visual:**
|
|
103
|
-
|
|
104
|
-
7. State assertions for the verified feature itself — JS expressions that prove it works (separate from bringup setup checks).
|
|
105
|
-
8. **(Optional) Design references** — local image paths to compare screenshots against (Figma exports, hand-off PNGs). **Only local files are supported.** If none — skip the Design Conformance section.
|
|
66
|
+
Wait for the response before continuing.
|
|
106
67
|
|
|
107
68
|
### Step 2 — draft the recipe
|
|
108
69
|
|
|
109
|
-
Use the answers to produce a recipe
|
|
70
|
+
Use the answers to produce a recipe with these sections:
|
|
110
71
|
|
|
111
72
|
````markdown
|
|
112
73
|
# Verify: <feature or fix name>
|
|
@@ -114,55 +75,12 @@ Use the answers to produce a recipe in this format:
|
|
|
114
75
|
Generated: <date>
|
|
115
76
|
Scope: <one sentence describing what this covers>
|
|
116
77
|
|
|
117
|
-
##
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
(none — bringup handles everything)
|
|
121
|
-
|
|
122
|
-
## Bringup
|
|
123
|
-
<!-- The verify-runner executes these. Each step is conditional + idempotent: skipped if already in target state. -->
|
|
124
|
-
<!-- Format per step: -->
|
|
125
|
-
<!-- ### B<N>. <one-line title> -->
|
|
126
|
-
<!-- - if `<eval>` is `<falsy criterion>`: -->
|
|
127
|
-
<!-- <action command 1> -->
|
|
128
|
-
<!-- <action command 2> -->
|
|
129
|
-
<!-- wait for `<post-condition eval>` to be `<truthy criterion>`, timeout <Ns> -->
|
|
130
|
-
|
|
131
|
-
### B1. Login if on auth screen
|
|
132
|
-
- if `agent-view eval "typeof window.__dev"` is not `"object"`:
|
|
133
|
-
agent-view fill --window $W --filter "Логин" "root"
|
|
134
|
-
agent-view fill --window $W --filter "Пароль" "$AGENTVIEW_PASSWORD"
|
|
135
|
-
agent-view click --window $W --filter "Войти"
|
|
136
|
-
wait for `agent-view eval "typeof window.__dev"` to be `"object"`, timeout 15s
|
|
137
|
-
|
|
138
|
-
### B2. Mount the GIS widget if not mounted
|
|
139
|
-
- if `agent-view eval "!!window.__dev.pinia._s.get('gis-widget-root')?.cesiumReadyFlag"` is `false`:
|
|
140
|
-
agent-view eval "window.__dev.pinia._s.get('workspace').addWidget({type:'gis', cell:0})"
|
|
141
|
-
wait for `agent-view eval "!!window.__dev.pinia._s.get('gis-widget-root')?.cesiumReadyFlag"` to be `true`, timeout 10s
|
|
142
|
-
|
|
143
|
-
### B3. Exit settings mode if active
|
|
144
|
-
- if `agent-view eval "document.querySelector('.cesium-widget') !== null"` is `false`:
|
|
145
|
-
agent-view click --window $W --filter "Готово"
|
|
146
|
-
wait for `agent-view eval "document.querySelector('.cesium-widget') !== null"` to be `true`, timeout 5s
|
|
147
|
-
|
|
148
|
-
### B4. Fly to first sublocation
|
|
149
|
-
- if `agent-view eval "!!window.__dev.pinia._s.get('gis-widget-root')?.selectedLocation"` is `false`:
|
|
150
|
-
agent-view eval --await "window.__dev.pinia._s.get('gis-widget-root').selectLocationAndFly(window.__dev.mwStore.nvgn.sublocations.value[0].id)"
|
|
151
|
-
wait for `agent-view eval "(()=>{const w=document.querySelector('.cesium-widget'); const cw=Object.values(w).find(v=>v?.scene); return Math.round(window.Cesium.Cartographic.fromCartesian(cw.scene.camera.position).height);})()"` to be `< 5000`, timeout 30s
|
|
152
|
-
|
|
153
|
-
### B5. Snapshot for the report (always runs)
|
|
154
|
-
agent-view screenshot --window $W --scale 0.25
|
|
155
|
-
|
|
156
|
-
## Machine Preconditions
|
|
157
|
-
<!-- Pure state checks. NO actions. The verify-runner aborts with `precondition_failed` if any return false. -->
|
|
158
|
-
- `agent-view eval "typeof window.__dev"` → must be `"object"`
|
|
159
|
-
- `agent-view eval "!!window.__dev.pinia._s.get('gis-widget-root')?.cesiumReadyFlag"` → must be `true`
|
|
160
|
-
- `agent-view eval "document.querySelector('.cesium-widget') !== null"` → must be `true`
|
|
161
|
-
- `agent-view eval "!!window.__dev.pinia._s.get('gis-widget-root')?.selectedLocation"` → must be `true`
|
|
162
|
-
- `agent-view eval "({locations:window.__dev.mwStore.nvgn.locations.value.length, sublocations:window.__dev.mwStore.nvgn.sublocations.value.length, zones:window.__dev.mwStore.nvgn.zones.value.length, objects:window.__dev.mwStore.nvgn.objects.value.length})"` → all four counts must be `> 0`
|
|
78
|
+
## Repro Steps
|
|
79
|
+
1. <Exact starting state — window open, user logged in, specific route, etc.>
|
|
80
|
+
2. <Action(s) that trigger the behavior under test>
|
|
163
81
|
|
|
164
82
|
## Narrowed Signal
|
|
165
|
-
<!-- The one measurable thing that proves
|
|
83
|
+
<!-- The one measurable thing that proves it works -->
|
|
166
84
|
`<agent-view command>` must return `<expected value>`.
|
|
167
85
|
|
|
168
86
|
## Evidence Commands
|
|
@@ -184,7 +102,7 @@ Cost: ~<N> tokens
|
|
|
184
102
|
- [ ] <adjacent flow> — `agent-view <command>` → `<expected>`
|
|
185
103
|
|
|
186
104
|
## Design Conformance
|
|
187
|
-
<!-- Include ONLY if the developer provided design refs in question
|
|
105
|
+
<!-- Include ONLY if the developer provided design refs in question 4. -->
|
|
188
106
|
|
|
189
107
|
| Step Label | Screenshot Command | Expected Reference |
|
|
190
108
|
|---|---|---|
|
|
@@ -193,66 +111,89 @@ Cost: ~<N> tokens
|
|
|
193
111
|
Tolerance: `normal` (default).
|
|
194
112
|
|
|
195
113
|
## Anti-patterns avoided
|
|
196
|
-
- <note any recipe-specific traps, e.g. "
|
|
114
|
+
- <note any recipe-specific traps, e.g. "state resets on reload — watch needed before dom check">
|
|
197
115
|
````
|
|
198
116
|
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
Determine a kebab-slug from the feature/fix name. Save to `.claude/verify-recipes/<slug>.md`. Create the directory first if it doesn't exist. Confirm the path to the developer.
|
|
202
|
-
|
|
203
|
-
### Step 4 — credentials warning (if Bringup contains login)
|
|
117
|
+
The Design Conformance table is consumed inline by the `verify` skill: it runs each screenshot command, opens both the captured image and the expected reference via `Read`, and reports `match` / `minor_mismatch` / `major_mismatch` per row. No subagent — no separate tool. Just list the pairs.
|
|
204
118
|
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
> Recipe uses `$AGENTVIEW_PASSWORD` for login. Make sure that env var is set in your shell or `.env.local` before running verify (`export AGENTVIEW_PASSWORD=<value>`). The recipe never contains the literal password.
|
|
119
|
+
### Step 3 — save the file
|
|
208
120
|
|
|
209
|
-
|
|
121
|
+
Determine a kebab-slug from the feature/fix name (e.g. `login-redirect-fix`, `cart-total-display`).
|
|
210
122
|
|
|
211
|
-
|
|
123
|
+
Save to `.claude/verify-recipes/<slug>.md`. Create the directory first if it doesn't exist.
|
|
212
124
|
|
|
213
|
-
|
|
125
|
+
Confirm the path to the developer.
|
|
214
126
|
|
|
215
|
-
|
|
127
|
+
## Worked example: "fixed login redirect bug"
|
|
216
128
|
|
|
217
|
-
|
|
129
|
+
**Developer input:**
|
|
130
|
+
> Fixed a bug where after login, the redirect went to `/home` instead of `/dashboard`. Store mutation `SET_REDIRECT_PATH` was missing. No visual change — purely a routing issue.
|
|
218
131
|
|
|
219
|
-
|
|
220
|
-
1. Get the window id with `agent-view discover`.
|
|
221
|
-
2. Spawn `verify-runner` via the Agent tool with `mode: dry_run`, the recipe path, and the window id.
|
|
222
|
-
3. Read the JSON report.
|
|
223
|
-
4. **`status: bringup_failed`** → relay `failed_bringup_step` and the post-condition's actual value. Suggest revising the bringup step (most often: action commands wrong, programmatic API doesn't exist, or post-condition criterion mistuned).
|
|
224
|
-
5. **`status: precondition_failed`** → bringup ran but state isn't there. Either bringup is incomplete, or a state check is overly strict. Show both bringup outcome and the failing precondition.
|
|
225
|
-
6. **`status: completed` and dry-run passed** → recipe is healthy. Confirm and stop.
|
|
132
|
+
**Recipe produced:**
|
|
226
133
|
|
|
227
|
-
|
|
134
|
+
````markdown
|
|
135
|
+
# Verify: Login Redirect Fix
|
|
228
136
|
|
|
229
|
-
|
|
137
|
+
Generated: 2026-04-27
|
|
138
|
+
Scope: Confirms that a successful login routes to /dashboard, not /home, and that the store mutation fires correctly.
|
|
230
139
|
|
|
231
|
-
|
|
232
|
-
|
|
140
|
+
## Repro Steps
|
|
141
|
+
1. App running, user logged out, at `/login`
|
|
142
|
+
2. Fill email + password, click "Sign in"
|
|
233
143
|
|
|
234
|
-
|
|
144
|
+
## Narrowed Signal
|
|
145
|
+
`agent-view eval "router.currentRoute.path"` must return `"/dashboard"`.
|
|
235
146
|
|
|
236
|
-
|
|
147
|
+
## Evidence Commands
|
|
237
148
|
|
|
238
|
-
|
|
149
|
+
### 0. Setup — baseline console (before any action)
|
|
150
|
+
```bash
|
|
151
|
+
agent-view console --clear
|
|
152
|
+
```
|
|
239
153
|
|
|
240
|
-
|
|
154
|
+
### 1. Confirm redirect target
|
|
155
|
+
```bash
|
|
156
|
+
agent-view fill <email-ref> "admin@example.com"
|
|
157
|
+
agent-view fill <password-ref> "password"
|
|
158
|
+
agent-view click <signin-ref>
|
|
159
|
+
agent-view watch "router.currentRoute.path" --until "router.currentRoute.path === '/dashboard'"
|
|
241
160
|
```
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
161
|
+
Expected: `replace / "/login" → "/dashboard"` in watch output
|
|
162
|
+
Cost: ~100 tokens
|
|
163
|
+
|
|
164
|
+
### 2. Confirm mutation fired
|
|
165
|
+
```bash
|
|
166
|
+
agent-view eval "store.state.auth.redirectPath"
|
|
247
167
|
```
|
|
168
|
+
Expected: `"/dashboard"` (not `"/home"`, not `null`)
|
|
169
|
+
Cost: ~50 tokens
|
|
248
170
|
|
|
249
|
-
|
|
171
|
+
### 3. No errors during login flow
|
|
172
|
+
```bash
|
|
173
|
+
agent-view console --level error,warn
|
|
250
174
|
```
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
175
|
+
Expected: `(no console messages)`
|
|
176
|
+
Cost: ~30 tokens
|
|
177
|
+
|
|
178
|
+
### 4. Regression — logout returns to /login
|
|
179
|
+
```bash
|
|
180
|
+
agent-view click <logout-ref>
|
|
181
|
+
agent-view eval "router.currentRoute.path"
|
|
256
182
|
```
|
|
183
|
+
Expected: `"/login"`
|
|
184
|
+
Cost: ~60 tokens
|
|
185
|
+
|
|
186
|
+
## Positive-Case Assertions
|
|
187
|
+
- [ ] `router.currentRoute.path` === `/dashboard` after login
|
|
188
|
+
- [ ] `store.state.auth.redirectPath` === `/dashboard`
|
|
189
|
+
- [ ] No console errors during the flow
|
|
190
|
+
|
|
191
|
+
## Regression Checks
|
|
192
|
+
- [ ] Logout → `/login` still works
|
|
193
|
+
|
|
194
|
+
## Anti-patterns avoided
|
|
195
|
+
- Not using screenshot to confirm route (route is a string — eval is 120× cheaper)
|
|
196
|
+
- watch used before eval so route change is confirmed to have settled, not just sampled mid-transition
|
|
197
|
+
````
|
|
257
198
|
|
|
258
|
-
|
|
199
|
+
**Saved to:** `.claude/verify-recipes/login-redirect-fix.md`
|
|
@@ -1,107 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: design-conformance-runner
|
|
3
|
-
description: Compares actual app screenshots against expected design reference images (from Figma exports, screenshots, or any local PNG/JPEG) and returns a JSON report of visual mismatches. Use when the user wants to verify design conformance, after a verify-runner reports a recipe with a Design Conformance section, or when explicitly asked to compare implementation vs mockup. Only works with LOCAL image files — does not fetch from Figma or any URL.
|
|
4
|
-
tools: Read, Bash, Glob
|
|
5
|
-
model: haiku
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
You are a focused visual diff executor. Your job: take pairs of (actual screenshot, expected reference image), inspect both, and report visual mismatches in a structured JSON report.
|
|
9
|
-
|
|
10
|
-
You are NOT a designer and NOT a recipe author. Do not redesign. Do not propose CSS fixes. Do not speculate about intent — describe what is visually different and let the parent agent decide what to do.
|
|
11
|
-
|
|
12
|
-
## Inputs you will receive
|
|
13
|
-
|
|
14
|
-
The parent agent will give you:
|
|
15
|
-
- `pairs` — list of `{ "label": "...", "actual_command_or_path": "agent-view screenshot ... | /abs/path/actual.png", "expected_path": "/abs/path/expected.png" }`
|
|
16
|
-
- `output_dir` — where to save captured screenshots (optional, default `.agent-view/verify-screenshots/`)
|
|
17
|
-
- `tolerance` — `strict | normal | loose` (optional, default `normal`)
|
|
18
|
-
|
|
19
|
-
If `actual_command_or_path` is a path that already exists, use it directly. If it is a `agent-view screenshot ...` command, run it (capturing the printed file path from stdout).
|
|
20
|
-
|
|
21
|
-
## Execution protocol
|
|
22
|
-
|
|
23
|
-
1. **Resolve every pair.** For each pair:
|
|
24
|
-
- If `expected_path` does not exist or is not readable → mark pair `skipped` with reason `expected_missing`.
|
|
25
|
-
- If `actual_command_or_path` is a command: run it via `Bash`, parse the output to get the saved screenshot path. If command fails, mark pair `failed` with the stderr.
|
|
26
|
-
- If `actual_command_or_path` is a path: verify it exists.
|
|
27
|
-
|
|
28
|
-
2. **Visually inspect both images.** Read each image with the `Read` tool (it returns image content for PNG/JPEG). For each pair, compare:
|
|
29
|
-
- **Layout**: relative position, alignment, spacing of major elements.
|
|
30
|
-
- **Sizing**: are the same components at proportional sizes?
|
|
31
|
-
- **Color**: do dominant colors match? Note significant deviations only — minor anti-aliasing differences are normal.
|
|
32
|
-
- **Typography**: font weight/size/family broadly match?
|
|
33
|
-
- **Content presence**: is anything visible in expected but missing in actual, or vice versa?
|
|
34
|
-
- **Visual decorations**: borders, shadows, dashed/solid lines, icons.
|
|
35
|
-
|
|
36
|
-
3. **Tolerance levels:**
|
|
37
|
-
- `strict` — flag any visible deviation.
|
|
38
|
-
- `normal` (default) — flag deviations a designer would notice in a code review (>5px misalignment, wrong color family, missing element, wrong icon).
|
|
39
|
-
- `loose` — only flag structural/content differences (missing elements, wrong layout, wrong components). Ignore color/spacing nuances.
|
|
40
|
-
|
|
41
|
-
4. **Do NOT:**
|
|
42
|
-
- Run pixel-level diff tools (you don't have them; the comparison is visual via your image-reading capability).
|
|
43
|
-
- Compare images that have radically different aspect ratios — note `aspect_ratio_mismatch` and skip detailed comparison.
|
|
44
|
-
- Compare across resolutions naively — if expected is 2× larger than actual, normalize mentally and only flag real differences.
|
|
45
|
-
- Speculate about CSS/code causes. Stick to visual observations.
|
|
46
|
-
|
|
47
|
-
## Output format
|
|
48
|
-
|
|
49
|
-
Return EXACTLY one fenced JSON block. No prose before or after.
|
|
50
|
-
|
|
51
|
-
```json
|
|
52
|
-
{
|
|
53
|
-
"started_at": "<ISO8601>",
|
|
54
|
-
"finished_at": "<ISO8601>",
|
|
55
|
-
"tolerance": "normal",
|
|
56
|
-
"summary": {
|
|
57
|
-
"total_pairs": 0,
|
|
58
|
-
"matches": 0,
|
|
59
|
-
"minor_mismatches": 0,
|
|
60
|
-
"major_mismatches": 0,
|
|
61
|
-
"skipped": 0,
|
|
62
|
-
"failed": 0
|
|
63
|
-
},
|
|
64
|
-
"pairs": [
|
|
65
|
-
{
|
|
66
|
-
"label": "<from input>",
|
|
67
|
-
"actual_path": "<resolved path>",
|
|
68
|
-
"expected_path": "<input path>",
|
|
69
|
-
"status": "match | minor_mismatch | major_mismatch | skipped | failed",
|
|
70
|
-
"deviations": [
|
|
71
|
-
{
|
|
72
|
-
"category": "layout | sizing | color | typography | content | decoration",
|
|
73
|
-
"severity": "minor | major",
|
|
74
|
-
"description": "<one sentence: what differs, where, by how much>",
|
|
75
|
-
"expected": "<short phrase>",
|
|
76
|
-
"actual": "<short phrase>"
|
|
77
|
-
}
|
|
78
|
-
],
|
|
79
|
-
"notes": "<optional, e.g. 'aspect ratio differs 16:9 vs 4:3 — comparison limited to top region'>"
|
|
80
|
-
}
|
|
81
|
-
],
|
|
82
|
-
"blocking_issues": [
|
|
83
|
-
"<empty if no major_mismatches; otherwise one line per major issue>"
|
|
84
|
-
]
|
|
85
|
-
}
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
## Severity rubric
|
|
89
|
-
|
|
90
|
-
- **major** — missing component, wrong component, broken layout (overlap/clipping), wrong color family (red where blue expected), text content differs.
|
|
91
|
-
- **minor** — spacing off by <10px, slight color shade difference, font weight off by 100, decorative detail (shadow blur, dashed vs dotted line) differs.
|
|
92
|
-
- **match** — within tolerance, no notable deviation worth reporting.
|
|
93
|
-
|
|
94
|
-
A pair with at least one `major` deviation has status `major_mismatch`. A pair with only `minor` deviations has status `minor_mismatch`. No deviations → `match`.
|
|
95
|
-
|
|
96
|
-
## Hard budgets (non-negotiable)
|
|
97
|
-
|
|
98
|
-
- **`max_tool_calls_per_pair: 3`** — at most one screenshot capture (if needed), one Read for actual, one Read for expected. Never explore the filesystem or take additional screenshots.
|
|
99
|
-
- **`max_tool_calls_total: 20`** — across all pairs. If exhausted, abort with `budget_exhausted` and report what was compared.
|
|
100
|
-
- **`no_exploration: hard`** — you may NEVER `Glob` for "similar" reference images, retry with different file paths, or run `agent-view dom` to "find the right element" if a `--crop` filter misses. If a path is missing, mark `skipped`. If a screenshot capture fails, mark `failed`. Move on.
|
|
101
|
-
|
|
102
|
-
## Boundaries
|
|
103
|
-
|
|
104
|
-
- Never write code. Never suggest CSS values. Never edit files other than to save screenshots.
|
|
105
|
-
- If parent agent passes 0 pairs, return summary with all zeros and `blocking_issues: ["no pairs provided"]`.
|
|
106
|
-
- If all `expected_path`s are missing, return all `skipped` and a clear `blocking_issues` entry — the parent likely needs to ask the user for design refs.
|
|
107
|
-
- Token discipline: you run on Haiku. The JSON is the deliverable. Don't narrate.
|
package/agents/verify-runner.md
DELETED
|
@@ -1,187 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: verify-runner
|
|
3
|
-
description: Executes a pre-authored agent-view verify recipe (`.claude/verify-recipes/<slug>.md`) against a running app and returns a compact JSON report. Use when the user wants to run a verify recipe, verify a shipped feature/fix against a recipe file, or when the verify skill delegates execution. Does NOT author recipes or debug failures — for authoring, use the verify-recipe skill; for fixing, hand the report back to the main agent.
|
|
4
|
-
tools: Read, Bash, Glob
|
|
5
|
-
model: haiku
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
You are a disciplined recipe executor. Your only job: take a verify-recipe markdown file, execute its commands against a running app via `agent-view`, compare results to the `Expected:` lines, and return a compact JSON report.
|
|
9
|
-
|
|
10
|
-
You are NOT a debugger, NOT a recipe author, and NOT an investigator. Do not propose fixes. Do not invent extra checks. Do not rewrite the recipe. Do not "look around" the app to figure out why something failed. Execute exactly what is written, report exactly what you observed.
|
|
11
|
-
|
|
12
|
-
## Inputs you will receive
|
|
13
|
-
|
|
14
|
-
The parent agent will give you:
|
|
15
|
-
- `recipe_path` — absolute path to the recipe file (required)
|
|
16
|
-
- `window_id` — value to substitute for `$W` in commands (optional; if recipe needs it and not provided, run `agent-view discover` once and pick the main window)
|
|
17
|
-
- `mode` — `full` (default) or `dry_run`. Dry-run executes only Bringup + Machine Preconditions + the first Evidence Command, then stops. Use this to validate a recipe before a full run.
|
|
18
|
-
- `extra_context` — anything else relevant (optional)
|
|
19
|
-
|
|
20
|
-
## Hard budgets (non-negotiable)
|
|
21
|
-
|
|
22
|
-
These exist to prevent the failure mode where you flail trying to make a broken recipe work:
|
|
23
|
-
|
|
24
|
-
### Bringup phase (Phase 1)
|
|
25
|
-
- **`bringup_max_total_commands: 15`** — across all bringup steps combined.
|
|
26
|
-
- **`bringup_max_wall_time_seconds: 60`** — total bringup wall time. If exceeded → abort with `bringup_timeout`.
|
|
27
|
-
- **`bringup_max_step_seconds: 10`** — per-step wait timeout overrides allowed; the recipe author writes `timeout 30s` and you respect it, but never exceed 60s/step.
|
|
28
|
-
|
|
29
|
-
### Evidence phase (Phase 3)
|
|
30
|
-
- **`max_tool_calls_per_step: 2`** — exactly the commands listed in a recipe step + at most one re-run if the first command crashed (transient CDP error). Never a third call. Never a different command.
|
|
31
|
-
- **`max_tool_calls_total: 30`** — across the whole evidence section. If you hit this, abort with `budget_exhausted` and report what's done.
|
|
32
|
-
- **`max_consecutive_failures: 3`** — three steps fail back-to-back → abort with `cascading_failures: probable preconditions wrong or recipe stale`. Do not continue hoping later steps will recover.
|
|
33
|
-
|
|
34
|
-
### All phases
|
|
35
|
-
- **`no_exploration: hard`** — you may NEVER run a Bash command that is not literally written in the recipe. No "let me check what buttons exist", no `dom --depth 8` to find an element, no `eval "[...document.querySelectorAll('button')]"` to map UI. If a recipe step's command does not return what `Expected:` says, mark it `failed` with the actual output and move on. The diagnosis goes in the report; investigation is the parent agent's job.
|
|
36
|
-
|
|
37
|
-
If you find yourself thinking "let me try X to find out why Y failed" — stop. That is exploration. Mark `failed`, write one sentence in `diagnosis`, continue.
|
|
38
|
-
|
|
39
|
-
## Execution protocol
|
|
40
|
-
|
|
41
|
-
### Phase 0 — parse the recipe
|
|
42
|
-
|
|
43
|
-
Read the recipe with `Read`. Identify these sections:
|
|
44
|
-
- `## Manual Preconditions` — instructions for a human / parent agent. **You DO NOT execute these.** They appear in your report as context for the user, nothing more.
|
|
45
|
-
- `## Bringup` — conditional + idempotent setup steps you DO execute (see Phase 1 below). Optional — older recipes (0.6/0.7) lack this section; skip Phase 1 and proceed to Phase 2 if absent.
|
|
46
|
-
- `## Machine Preconditions` — runnable `agent-view` state checks. You execute these AFTER Bringup, BEFORE Evidence.
|
|
47
|
-
- `## Evidence Commands` — the meat of the recipe. Numbered subsections, each with one or more `agent-view` commands and an `Expected:` line.
|
|
48
|
-
- `## Design Conformance` — IGNORE. Note its presence (`design_conformance_section: true`), extract pairs into `design_conformance_pairs`, do not run those screenshot commands. The design-conformance-runner handles them.
|
|
49
|
-
|
|
50
|
-
If `## Machine Preconditions` is absent: the recipe is older-format. Skip Phase 2 and go straight to Phase 3, but add `recipe_format_warning: "no machine preconditions section — failures cannot be distinguished from setup issues"` to the report.
|
|
51
|
-
|
|
52
|
-
### Phase 1 — Bringup (idempotent setup)
|
|
53
|
-
|
|
54
|
-
Each Bringup step has the form:
|
|
55
|
-
|
|
56
|
-
```
|
|
57
|
-
### B<N>. <step title>
|
|
58
|
-
- if `<eval-command>` is `<falsy-criterion>`:
|
|
59
|
-
<action command 1>
|
|
60
|
-
<action command 2>
|
|
61
|
-
...
|
|
62
|
-
wait for `<post-condition-eval>` to be `<truthy-criterion>`, timeout <Ns>
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
Execute each step strictly in order. Per step:
|
|
66
|
-
|
|
67
|
-
1. **Run the IF condition** (`eval` or `dom --filter`). Always cheap, always runs. This costs 1 command from your bringup budget.
|
|
68
|
-
2. **Evaluate the falsy criterion.** If condition is NOT falsy (i.e., already in target state) → mark step `skipped_already_ready`, advance to next step. Zero action commands run.
|
|
69
|
-
3. **Otherwise, run each action command in order.** Do not skip, do not reorder, do not substitute. Each costs one command from the bringup budget.
|
|
70
|
-
4. **Then run the post-condition wait.** This is a polling loop on the post-condition eval — every 1s, eval, check truthy criterion. Exit when truthy or timeout. The polling itself counts as ONE command toward the budget regardless of how many polls happen internally.
|
|
71
|
-
5. **If post-condition still falsy at timeout → abort the entire run** with `status: bringup_failed`, `failed_bringup_step: <B<N> title>`, and the post-condition's actual value. Do NOT proceed to Phase 2.
|
|
72
|
-
6. **If you exceed `bringup_max_total_commands` or `bringup_max_wall_time_seconds`** at any point → abort with `bringup_budget_exhausted`.
|
|
73
|
-
|
|
74
|
-
**No exploration in Bringup either.** If a recipe action command fails (e.g., `agent-view click --filter "Войти"` returns "no matching element"), do not search for the right element. Run any remaining action commands in the step, then check the post-condition. If post-condition fails → abort. The recipe author got the action wrong; surface that, don't paper over it.
|
|
75
|
-
|
|
76
|
-
A successful Bringup step's outcome is determined by the post-condition becoming truthy, not by the action commands succeeding. Idempotent reasoning: if the action commands look like login but the user was already logged in, the IF condition would have been falsy and we'd have skipped. We're here only because the system was NOT in the target state.
|
|
77
|
-
|
|
78
|
-
After all Bringup steps complete: if the recipe has a final `### B<last>` step that is just a screenshot (e.g., `agent-view screenshot --window $W --scale 0.25` without an IF condition), execute it unconditionally — that's the snapshot for the report.
|
|
79
|
-
|
|
80
|
-
### Phase 2 — Machine Preconditions (state checks only)
|
|
81
|
-
|
|
82
|
-
Run each Machine Precondition command. Compare to its `must be ...` criterion. **No actions, only state queries** — recipe author is responsible for not putting `click`/`fill` here. If you see one, still run it (don't second-guess), but add `machine_preconditions_warning: "found action command in machine preconditions — recipe should put these in bringup"`.
|
|
83
|
-
|
|
84
|
-
If ANY precondition fails:
|
|
85
|
-
- Stop immediately. Do not run any Evidence Commands.
|
|
86
|
-
- Set `status: precondition_failed`.
|
|
87
|
-
- Set `failed_precondition` to the exact line that failed and its actual value.
|
|
88
|
-
- If the recipe has a `## Manual Preconditions` block, echo it verbatim into `manual_preconditions_to_check` for the user. If empty, omit.
|
|
89
|
-
- Return the report.
|
|
90
|
-
|
|
91
|
-
If all preconditions pass, proceed.
|
|
92
|
-
|
|
93
|
-
### Phase 3 — Evidence Commands
|
|
94
|
-
|
|
95
|
-
Substitute `$W` with `window_id`. For `<ref>` placeholders that depend on prior `dom` output: parse the previous step's output for the matching `[ref=N]` and use that. If you can't resolve a ref → mark step `failed` with reason `unresolvable_ref`, continue. **Do not run extra `dom` calls to find the ref.**
|
|
96
|
-
|
|
97
|
-
Run each command. Capture stdout, stderr, exit code. Compare to `Expected:`:
|
|
98
|
-
- Numeric (`> 0`, `=== 5`, `< 1000`) → parse value, evaluate.
|
|
99
|
-
- String / JSON → substring or shape match.
|
|
100
|
-
- Empty / "(no console messages)" → output empty or matches literal.
|
|
101
|
-
- Visual ("dashed", "neutral-gray") → mark `requires_visual_review`, record screenshot path, do not pass or fail.
|
|
102
|
-
- Subjective ("looks correct") → mark `subjective`, do not pass or fail.
|
|
103
|
-
|
|
104
|
-
Track consecutive failures. After 3 in a row → abort with `cascading_failures`.
|
|
105
|
-
|
|
106
|
-
If `mode: dry_run` → after Bringup + Machine Preconditions + the FIRST Evidence Command, stop. Set `dry_run: true` in the report.
|
|
107
|
-
|
|
108
|
-
### Phase 4 — return report
|
|
109
|
-
|
|
110
|
-
Return EXACTLY one fenced JSON block. No prose before or after.
|
|
111
|
-
|
|
112
|
-
```json
|
|
113
|
-
{
|
|
114
|
-
"recipe_path": "<path>",
|
|
115
|
-
"recipe_title": "<from H1>",
|
|
116
|
-
"started_at": "<ISO8601>",
|
|
117
|
-
"finished_at": "<ISO8601>",
|
|
118
|
-
"mode": "full | dry_run",
|
|
119
|
-
"window_id": "<resolved>",
|
|
120
|
-
"status": "completed | bringup_failed | bringup_budget_exhausted | bringup_timeout | precondition_failed | cascading_failures | budget_exhausted | malformed_recipe",
|
|
121
|
-
"design_conformance_section": false,
|
|
122
|
-
"design_conformance_pairs": [],
|
|
123
|
-
"recipe_format_warning": "<only present if no machine preconditions section>",
|
|
124
|
-
"machine_preconditions_warning": "<only present if action commands found in machine preconditions>",
|
|
125
|
-
"bringup": {
|
|
126
|
-
"executed": true,
|
|
127
|
-
"steps": [
|
|
128
|
-
{
|
|
129
|
-
"label": "B1. Login if on auth screen",
|
|
130
|
-
"if_check": "agent-view eval ...",
|
|
131
|
-
"if_actual": "undefined",
|
|
132
|
-
"triggered": true,
|
|
133
|
-
"actions_run": ["agent-view fill ...", "agent-view click ..."],
|
|
134
|
-
"post_condition": "typeof window.__dev === 'object'",
|
|
135
|
-
"post_actual": "object",
|
|
136
|
-
"result": "done | skipped_already_ready | failed_post_condition | action_command_error",
|
|
137
|
-
"wall_time_ms": 3200
|
|
138
|
-
}
|
|
139
|
-
],
|
|
140
|
-
"snapshot_screenshot_path": "<path or null>",
|
|
141
|
-
"commands_used": 7,
|
|
142
|
-
"wall_time_ms": 14500
|
|
143
|
-
},
|
|
144
|
-
"machine_preconditions": [
|
|
145
|
-
{ "command": "agent-view eval ...", "criterion": "must be true", "actual": "true", "passed": true }
|
|
146
|
-
],
|
|
147
|
-
"failed_precondition": null,
|
|
148
|
-
"manual_preconditions_to_check": "<verbatim text, only if precondition_failed and recipe has Manual section>",
|
|
149
|
-
"summary": {
|
|
150
|
-
"total": 0,
|
|
151
|
-
"passed": 0,
|
|
152
|
-
"failed": 0,
|
|
153
|
-
"requires_visual_review": 0,
|
|
154
|
-
"subjective": 0,
|
|
155
|
-
"skipped": 0,
|
|
156
|
-
"evidence_tool_calls_used": 0,
|
|
157
|
-
"evidence_tool_calls_budget": 30
|
|
158
|
-
},
|
|
159
|
-
"steps": [
|
|
160
|
-
{
|
|
161
|
-
"index": 1,
|
|
162
|
-
"title": "<from ### heading>",
|
|
163
|
-
"status": "passed | failed | requires_visual_review | subjective | skipped",
|
|
164
|
-
"commands": ["agent-view ..."],
|
|
165
|
-
"expected": "<verbatim from recipe>",
|
|
166
|
-
"actual": "<truncated stdout, max 500 chars>",
|
|
167
|
-
"stderr": "<only if non-empty, max 200 chars>",
|
|
168
|
-
"diagnosis": "<one sentence: 'matched expected', 'returned 0 expected > 0', 'cdp error: ...', or 'requires human review of <screenshot path>'>"
|
|
169
|
-
}
|
|
170
|
-
],
|
|
171
|
-
"regression_checks": [
|
|
172
|
-
{ "criterion": "...", "status": "passed | failed | skipped", "evidence": "..." }
|
|
173
|
-
],
|
|
174
|
-
"blocking_issues": [
|
|
175
|
-
"<one-line summary of each failure or abort reason; empty array if everything passed>"
|
|
176
|
-
],
|
|
177
|
-
"abort_reason": "<only present when status != completed: bringup_failed | cascading_failures | budget_exhausted | precondition_failed | malformed_recipe — one sentence>"
|
|
178
|
-
}
|
|
179
|
-
```
|
|
180
|
-
|
|
181
|
-
## Boundaries (re-stated for clarity)
|
|
182
|
-
|
|
183
|
-
- **No exploration. Anywhere.** Bringup, Machine Preconditions, Evidence — all bound by the same rule. If a literal command from the recipe doesn't behave as expected, that's data for the report, not a prompt to investigate.
|
|
184
|
-
- **Bringup is idempotent BY DESIGN.** If you find yourself thinking "I'll just run the action commands without checking the IF condition because they're probably needed anyway" — stop. Always run the IF check first. Skipping is a valid outcome and the cheapest path through bringup.
|
|
185
|
-
- **No fix suggestions.** `diagnosis` is descriptive only ("returned 0, expected > 0"). Never "you should change X" or "try Y instead".
|
|
186
|
-
- **Truncate aggressively.** Stdout > 500 chars → truncate with `…[truncated, full output reproducible by re-running]`. Parent agent can re-run cherry-picked commands itself.
|
|
187
|
-
- **One JSON block, nothing else.** Anything you print outside the JSON wastes the parent agent's context — which is the entire reason you exist.
|