@petukhovart/agent-view 0.13.0 → 0.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/.claude-plugin/plugin.json +2 -2
  3. package/README.md +52 -1
  4. package/dist/cdp/transport.d.ts.map +1 -1
  5. package/dist/cdp/transport.js +128 -4
  6. package/dist/cdp/transport.js.map +1 -1
  7. package/dist/cdp/types.d.ts +51 -0
  8. package/dist/cdp/types.d.ts.map +1 -1
  9. package/dist/cdp/types.js.map +1 -1
  10. package/dist/cli/commands/click.d.ts +1 -0
  11. package/dist/cli/commands/click.d.ts.map +1 -1
  12. package/dist/cli/commands/click.js +2 -0
  13. package/dist/cli/commands/click.js.map +1 -1
  14. package/dist/cli/commands/coverage.d.ts +14 -0
  15. package/dist/cli/commands/coverage.d.ts.map +1 -0
  16. package/dist/cli/commands/coverage.js +34 -0
  17. package/dist/cli/commands/coverage.js.map +1 -0
  18. package/dist/cli/commands/listeners.d.ts +11 -0
  19. package/dist/cli/commands/listeners.d.ts.map +1 -0
  20. package/dist/cli/commands/listeners.js +34 -0
  21. package/dist/cli/commands/listeners.js.map +1 -0
  22. package/dist/cli/index.js +30 -0
  23. package/dist/cli/index.js.map +1 -1
  24. package/dist/inspectors/coverage/index.d.ts +17 -0
  25. package/dist/inspectors/coverage/index.d.ts.map +1 -0
  26. package/dist/inspectors/coverage/index.js +65 -0
  27. package/dist/inspectors/coverage/index.js.map +1 -0
  28. package/dist/inspectors/listeners/index.d.ts +4 -0
  29. package/dist/inspectors/listeners/index.d.ts.map +1 -0
  30. package/dist/inspectors/listeners/index.js +30 -0
  31. package/dist/inspectors/listeners/index.js.map +1 -0
  32. package/dist/server/server.d.ts +13 -5
  33. package/dist/server/server.d.ts.map +1 -1
  34. package/dist/server/server.js +174 -64
  35. package/dist/server/server.js.map +1 -1
  36. package/package.json +1 -1
  37. package/skills/verify/SKILL.md +157 -421
  38. package/skills/verify/references/commands.md +358 -0
  39. package/skills/verify/references/design-conformance.md +23 -0
@@ -1,466 +1,202 @@
1
1
  ---
2
2
  name: verify
3
- description: "Visual + runtime verification of desktop apps via Chrome DevTools Protocol. Use when modifying UI components, fixing visual bugs, testing user interactions, verifying layout, or when any workflow phase needs to inspect the running application — DOM, screenshots, scene graph, runtime state in pages and SharedWorkers/ServiceWorkers, console errors, or reactive-state diffs over time. Triggers on: verify, check UI, test how it looks, visual regression, screenshot, inspect DOM, check store/state, watch state changes, what changed after click, wait until state, read worker, console errors, runtime check, eval in page."
3
+ description: "Visual and runtime verification of a running app over CDP. Use when user looking for: checking a UI change, reproducing a visual bug, driving interactions, or when any workflow phase must inspect the live app — DOM, screenshots, store and worker state, console errors, network calls, or which code an action actually reached, or mention: verify, agent-view, check UI, visual regression"
4
4
  allowed-tools: Bash(agent-view *), Read
5
5
  ---
6
6
 
7
7
  # Visual Verification with agent-view
8
8
 
9
- You have access to `agent-view` CLI for inspecting and interacting with desktop applications via Chrome DevTools Protocol.
9
+ `agent-view` inspects and drives a running desktop app over Chrome DevTools Protocol.
10
10
 
11
11
  ## Prerequisites
12
12
 
13
- The target project must have:
14
- 1. `agent-view.config.json` in project root (run `agent-view init` to generate)
15
- 2. CDP enabled in the app (e.g. `--remote-debugging-port=9876` for Electron — avoid `9222`, it's Chrome's own default and collides when Chrome is open)
13
+ The target project needs `agent-view.config.json` in its root (`agent-view init` writes one) and CDP enabled in the
14
+ app — e.g. `--remote-debugging-port=9876` for Electron. Avoid `9222`: it is Chrome's own default and collides when
15
+ Chrome is open. The server is lazy: it starts on the first call and shuts down after 5 min idle.
16
16
 
17
- If config is missing, run `agent-view init` first.
17
+ ## Commands
18
18
 
19
- ## Commands Reference
19
+ Flags, output contracts and per-command failure modes live in **[`references/commands.md`](references/commands.md)** —
20
+ read it before your first call in a session.
20
21
 
21
- ### Discovery & Launch
22
- ```bash
23
- agent-view launch # Start app from config, wait for CDP readiness (all runtimes incl. Tauri)
24
- agent-view discover # List windows (JSON) — get window IDs
25
- agent-view stop # Stop the lazy server
26
- ```
27
-
28
- **Port conflict**: if the configured port is held by a non-CDP process (e.g. a stray webpack-dev on the same port), `agent-view launch` exits non-zero with `code: PORT_CONFLICT` and reports the owning PID/process name. Ask the user to either close that process or start the app manually — do not kill foreign processes from the skill.
29
-
30
- **Tauri**: launch works the same as Electron, but the wait timeout is 10 min (cargo builds are slow). If you see a port conflict on the Tauri devUrl port, it's almost always a parallel browser-dev — surface the PID and ask the user.
31
-
32
- ### DOM Inspection
33
- ```bash
34
- agent-view dom # DOM accessibility tree (default window)
35
- agent-view dom --window <id|name> # Specific window
36
- agent-view dom --filter "button" # Filter by text/role
37
- agent-view dom --depth 3 # Limit tree depth
38
- agent-view dom --compact # Merge single-child chains onto one line (~40-60% fewer tokens)
39
- agent-view dom --count # Count of all visible nodes (single integer line)
40
- agent-view dom --filter "row" --count # Count matching nodes — e.g. "does this table have 5 rows?"
41
- agent-view dom --max-lines 200 # Hard line budget; refs for truncated nodes still stored
42
- agent-view dom --diff # Lines changed since last dom call (+ added / - removed)
43
- ```
44
-
45
- `--count` skips tree output and ref mutations — cheapest way to assert "element exists N times" without loading the full tree into context.
46
-
47
- ### Interaction
48
- ```bash
49
- agent-view click <ref> # Click element by ref from dom output
50
- agent-view click --filter "Save" # Find element by text and click
51
- agent-view click --pos 100,200 # Click by coordinates — CANVAS ONLY, see below
52
- agent-view click <ref> --double # Double-click (fires dblclick handlers); works with --filter / --pos too
53
- agent-view fill <ref> "text" # Type into input field
54
- agent-view drag --from <ref> --to <ref> # Drag element to another element by ref
55
- agent-view drag --from-pos 50,80 --to-pos 200,300 # Drag by coordinates (for canvas / Pixi)
56
- agent-view drag --from <ref> --to <ref> --steps 25 --hold-ms 60 # Smoother movement, longer hold
57
- ```
58
-
59
- `drag` dispatches `mousePressed` → N × `mouseMoved` → `mouseReleased` via CDP. Endpoints can mix
60
- ref and coordinate (e.g. `--from <ref> --to-pos 400,300`). For canvas/Pixi targets always use
61
- `--from-pos`/`--to-pos` — derive the centroid via `agent-view eval` from the scene graph.
62
- Refs are resolved fresh on each call, so window resizes between snapshots are tolerated.
63
- Increase `--steps` for handlers using `globalpointermove` so intermediate frames are not skipped.
64
-
65
- **Coordinates are a last resort.** `--pos` / `--from-pos` / `--to-pos` exist for canvas and WebGL,
66
- where no ref exists. On DOM the correct order is `dom --filter "<text>"` → take the `[ref=N]` →
67
- `click <ref>`, or `click --filter "<text>"` in one step. A coordinate pair breaks on any layout
68
- shift, scroll, zoom, or window resize, and it clicks whatever now sits at that point — silently.
69
- If you reach for `--pos` on a DOM element, first say why the ref was not usable.
70
-
71
- ### Modals & file pickers (`dialog`, `upload`)
72
-
73
- A modal that agent-view cannot answer stops a run dead: the window looks frozen, every
74
- later command times out, and nothing says why. Two kinds, handled differently.
75
-
76
- **JS modals — `alert` / `confirm` / `prompt` / `beforeunload` — are answered for you.**
77
- Nothing to set up. While agent-view is attached these never block the page: the default
78
- standing answer is *dismiss*, and each one is recorded. `agent-view dialog` is the
79
- reliable place to read that record; the console feed carries the same line
80
- (`[agent-view] confirm auto-dismissed: <message>`) but only from the moment console
81
- attaches, so a modal answered before your first `console` call reaches `dialog` alone.
82
-
83
- ```bash
84
- agent-view dialog # standing answer + every modal this window has seen
85
- agent-view dialog policy accept # confirm() → true from now on
86
- agent-view dialog policy accept --text "name" # prompt() → "name"
87
- agent-view dialog policy dismiss # back to the default
88
- agent-view dialog dismiss # answer one that is open right now
89
- agent-view dialog accept --text "x" # …ditto, accepting
90
- ```
91
-
92
- `dialog accept` / `dialog dismiss` exist for a modal that was already open **before**
93
- agent-view attached — that one produced no event, so no policy applied to it.
94
-
95
- **Native file pickers never open — you answer them in advance.** Which mechanism applies
96
- depends on how the app opens the picker, and `dialog arm` sets up all of them at once,
97
- so you do not have to know:
98
-
99
- ```bash
100
- # The input already exists in the DOM (even hidden) — no picker at all, cheapest path
101
- agent-view upload --selector "#file-input" --file ./fixtures/a.png
102
- agent-view upload --selector "#imgs" --file ./a.png --file ./b.png # multi-select
103
- agent-view upload --ref 12 --file ./a.png # if the AX tree exposes it
104
-
105
- # The input is created inside the click handler, or the app calls a native dialog API
106
- agent-view dialog arm --file ./fixtures/a.png # then click the button that opens it
107
- agent-view dialog arm --cancel # act as if the user pressed Cancel
108
- agent-view dialog disarm # let real pickers open again
109
- ```
110
-
111
- Rules that actually bite:
112
-
113
- - **Arm before the click.** A picker cannot be caught once it is open. `arm` is one-shot —
114
- it is spent by the first picker and interception turns itself off, so a later click
115
- opens a real OS dialog.
116
- - **Click through `agent-view click`, never `eval "el.click()"`.** Chromium refuses to
117
- open a file picker without user activation, and an eval-driven click carries none: the
118
- picker is silently dropped and never intercepted.
119
- - **Hidden inputs have no ref.** `display:none` / `v-show="false"` keeps them out of the
120
- AX tree, so `dom` never prints one. Use `--selector`. `upload` has no `--filter` on
121
- purpose: an accessible name lands on the label, not on the input behind it.
122
- - **`beforeunload` is always dismissed**, whatever the policy — accepting it navigates
123
- away and loses the state you are checking.
124
- - Paths are resolved against your cwd and must exist — CDP accepts a bad path silently and
125
- the app then reads an empty file.
126
- - `agent-view dialog` after the fact shows what was intercepted and what was answered.
127
-
128
- Known limits: `showOpenFilePicker()` (File System Access API) exposes no input to fill, so
129
- it can only be cancelled. Electron does not implement `window.prompt` at all. Native
130
- dialogs opened straight from an Electron **main** process (`dialog.showOpenDialog` behind
131
- an IPC channel) are out of reach — CDP does not see the main process.
132
-
133
- ### Waiting (`wait`)
134
-
135
- ```bash
136
- agent-view wait --filter "Saved" # until the text appears in the AX tree
137
- agent-view wait --filter "Saved" --timeout 20 # max wait in seconds (default 10)
138
- agent-view wait --filter "Row 5" --window "Main" # specific window
139
- ```
140
-
141
- Exits as soon as the element appears; exits non-zero on timeout — so `&&` after it is a real gate.
142
-
143
- **Never sleep for a fixed time.** No `sleep`, no `timeout`, no `ping -n N 127.0.0.1` as a delay
144
- (agents reach for `ping` when the harness blocks `sleep` — it is the same mistake in worse
145
- clothing). A fixed pause is either too short, and you assert against a half-rendered UI, or too
146
- long, and you burn wall-clock on every step. agent-view has a condition-based wait for every kind
147
- of signal:
148
-
149
- | You are waiting for… | Use |
150
- |---|---|
151
- | An element to render | `wait --filter "<text>"` |
152
- | A store/state value | `watch "<expr>" --until "<expr>"` |
153
- | A log line | `console --follow --until "<pattern>"` |
154
- | A request to fire | `network --follow --until "<url>"` |
155
-
156
- If none of these fits, poll: `dom --filter X --count` in a loop with an explicit attempt cap, and
157
- report how many attempts it took.
158
-
159
- ### Screenshots
160
- ```bash
161
- agent-view screenshot --scale 0.5 # Recommended: JPEG at half-res (~3× fewer vision tokens)
162
- agent-view screenshot --scale 0.5 --window <id> # Specific window
163
- agent-view screenshot --crop "Sidebar" # Crop to element bounding box (~1.6k tokens — 12× win)
164
- agent-view screenshot --crop "Chart" --scale 0.5 # Crop + scale (stacks)
165
- agent-view screenshot --crop "Active bookings" --crop-up 1 # Crop the card, not just its heading
166
- agent-view screenshot # Full-res PNG (expensive: ~19k tokens at 1920×1080)
167
- ```
168
-
169
- `--crop <filter>` resolves the element with the same filter syntax as `dom --filter`, then crops the screenshot to its bounding box. Prefer `--crop` over full-window screenshots whenever you only need to inspect a specific section. Falls back to full-window with a stderr warning if the filter matches nothing.
170
-
171
- A text filter usually matches the text-bearing node, so cropping on a section title returns a thin strip of that title. `--crop-up <n>` climbs `n` element ancestors before cropping — use `1` (sometimes `2`) to get the surrounding card/section. When a crop comes back text-sized, the command says so on stderr.
172
-
173
- ### Runtime State (`eval`)
174
-
175
- Reads runtime values DOM/screenshot can't reveal — store contents, computed flags, worker internals.
176
- **Requires `"allowEval": true` in `agent-view.config.json`** — if the call returns "eval is disabled", tell the user to add the flag rather than working around it.
177
-
178
- ```bash
179
- agent-view eval "store.state.user.role" # default page target
180
- agent-view eval --window "Settings" "router.currentRoute.path"
181
- agent-view eval --target sync-worker "self.queue.length" # SharedWorker / ServiceWorker by id or substring
182
- agent-view eval --await "fetch('/api/health').then(r => r.status)"
183
- agent-view eval --json "({ buttons: document.querySelectorAll('button').length })"
184
- ```
22
+ | Commands | What they cover |
23
+ |--------------------------------------|----------------------------------------------------------------|
24
+ | `launch` `discover` `stop` `targets` | start the app; list windows and worker targets |
25
+ | `dom` | accessibility tree — filter, count, diff, depth cap |
26
+ | `click` `fill` `drag` | interaction by ref, by text, or by coordinate |
27
+ | `wait` | block until an element appears; non-zero on timeout |
28
+ | `dialog` `upload` | JS modals and native file pickers |
29
+ | `screenshot` | full window, scaled, or cropped to one element |
30
+ | `eval` `watch` | state now / state trajectory over time (both need `allowEval`) |
31
+ | `console` `logs` | message ring buffer / durable file feed that survives reloads |
32
+ | `network` | requests, headers, timing, bodies, WebSocket frames |
33
+ | `coverage` `listeners` | which functions ran; what handler is bound to a node |
34
+ | `scene` `snap` | canvas / WebGL scene graph (only when `webgl` is configured) |
185
35
 
186
- When to reach for `eval` instead of `dom`:
187
- - The truth lives in JS state, not the DOM (Pinia/Vuex/Redux/Zustand store, Vue refs, computed values, app singletons).
188
- - The target is a worker (`shared_worker`, `service_worker`, `worker`) — DOM doesn't exist there.
189
- - You need a precise number/string answer, not a tree to scan.
190
- - Verifying a `window.*` API or globally-exposed object exists. `eval` runs in the page's **main world**, so anything set on `window` directly or exposed via `contextBridge.exposeInMainWorld` is reachable. APIs placed only in an isolated-world preload (without `contextBridge`) will NOT be visible — that is not an agent bug, that is the host app's wiring.
191
-
192
- ### Reactive State (`watch`)
193
-
194
- Streams JSON-patch diffs of an expression over time. Use when you need to see *what changed* between an action and a final state — `eval` shows the snapshot, `watch` shows the trajectory. **Requires `"allowEval": true`.**
195
-
196
- ```bash
197
- agent-view watch "store.cart.total" # 250ms poll, default 10 changes or 30s
198
- agent-view watch "appState" --until "appState.status === 'ready'" # wait-for-condition with diff log
199
- agent-view watch "store.user" --max-changes 1 # capture exactly one change after a click
200
- agent-view watch "appState" --json # NDJSON, machine-readable
201
- ```
202
-
203
- When to reach for `watch` instead of `eval`:
204
- - Debugging "the click did X but state shows Y — what happened in between?"
205
- - Time-based assertions ("wait until store.status === 'ready'") — `--until` exits cleanly when truthy.
206
- - Confirming an action triggered the *expected* sequence of mutations, not just the final state.
207
-
208
- Output: `init` line (baseline), one line per RFC 6902 op (`replace /path old → new`, `add /items/0 ...`), final `stop` line with reason. Snapshot size cap 256 KB — narrow the expression (`store.x.y`, not `store`) for large objects.
209
-
210
- ### Console (`console`)
211
-
212
- Streams `Runtime.consoleAPICalled` + `Log.entryAdded`. Use to confirm a flow finished without errors, or to surface a specific warning after an interaction.
213
-
214
- ```bash
215
- agent-view console # buffered messages from auto-attached targets
216
- agent-view console --level error,warn # filter
217
- agent-view console --target sync-worker # one target (title/URL substring, same fuzzy semantics as eval --target)
218
- agent-view console --target IJ56KL # one target (exact id)
219
- agent-view console --follow --timeout 10 # stream window (use sparingly — 10s of waiting)
220
- agent-view console --follow --until "ready" # exit as soon as a message contains "ready"
221
- agent-view console --follow --until "/error/i" # exit on regex match (case-insensitive)
222
- agent-view console --clear # baseline before an interaction
223
- agent-view console --since "2026-04-26T10:00:00Z"
224
- ```
225
-
226
- `--until` requires `--follow`. Exits immediately when a message matches (substring or `/regex/flags`). On timeout without match exits non-zero.
227
-
228
- Standard pattern for "did this action error?":
229
- ```bash
230
- agent-view console --clear
231
- agent-view click --filter "Save"
232
- agent-view wait --filter "Saved"
233
- agent-view console --level error # expect "(no console messages)"
234
- ```
235
-
236
- Default attached target types: `page`, `shared_worker`, `service_worker` (override via `consoleTargets` in config).
237
-
238
- ### Log feed (`logs`)
239
-
240
- `console` reads a ring buffer that dies with the server; `logs` records the same messages — page *and* every worker — into one file you can grep later. Reach for it when a bug needs a timeline instead of a snapshot: intermittent failures, long scenarios, anything spanning reloads or worker restarts.
241
-
242
- ```bash
243
- agent-view logs start --truncate # start clean; keeps recording across reloads
244
- agent-view logs # tail last 200 records (alias of `logs tail`)
245
- agent-view logs tail --grep "ws closed" # substring or /regex/
246
- agent-view logs tail --since -2m # -30s | -5m | -2h | 09:31 | 09:31:02.500 | ISO
247
- agent-view logs tail --level error,warn -n 50
248
- agent-view logs clear # truncate feed + drop console buffer (baseline)
249
- agent-view logs status # attached targets, feed size, tick count
250
- agent-view logs stop
251
- ```
252
-
253
- Feed format — one record per line, always `HH:MM:SS.mmm [level] [type:id8] text`, local time, embedded newlines escaped as `\n`. That makes external `grep`/`awk` safe on it; a wrapped stack or JSON payload never breaks line-oriented filtering.
254
-
255
- Default file `.agent-view/console.log` in the project root (override with `logFile` in config or `--file`). Caps at 8 MB, then rotates once to `<file>.prev` (`logMaxBytes` to change). Recording suspends the server's 5-min idle shutdown, so a long scenario keeps writing.
256
-
257
- Standard pattern for "it fails once every N runs":
258
- ```bash
259
- agent-view logs start --truncate
260
- # … drive the scenario, reload, retry as many times as needed …
261
- agent-view logs tail --level error,warn
262
- agent-view logs tail --grep "/socket|retry/" --since -5m
263
- agent-view logs stop
264
- ```
265
-
266
- **Probes** (`--probe <file.js>[@target]`, requires `allowEval`) inject JS that logs into the same feed — use it when the evidence you need isn't logged by the app (wrap a method, count events, dump a scheduler). The probe is re-injected automatically whenever its context is gone: page reload, worker restart. Write it idempotent and let it report via plain `console.log`.
267
-
268
- ```bash
269
- agent-view logs start --probe ./probes/orchestrator.js@shared_worker --probe ./probes/audio.js@index.html
270
- ```
271
-
272
- ### Network (`network`)
273
-
274
- Request/response timeline, headers, timing, bodies, and WebSocket/SSE frames. Use to confirm an expected API call fired, diagnose a silent 404 / CORS block / missing auth header, or verify "button disabled until the network completes".
275
-
276
- ```bash
277
- agent-view network # recent requests, newest at the bottom
278
- agent-view network --req 3 # expand one: headers, timing, body / WS frame log
279
- agent-view network --status 4xx,5xx # only failures (class or exact code, e.g. 404)
280
- agent-view network --method POST # mutations among reads
281
- agent-view network --type xhr,fetch # drop document/image/font noise
282
- agent-view network --url "*/api/save*" # URL substring or * glob
283
- agent-view network --follow --until "/api/save" # stream until a matching request fires
284
- agent-view network --clear # baseline before an interaction
285
- ```
286
-
287
- **Eager, unlike `console`.** `network` captures from app launch, so page-load traffic (initial XHR/fetch, auth handshakes, boot 404s) is usually already buffered by the time you call it — in most cases you don't need to reload. `console` is the opposite (lazy: attaches on first call, loses earlier output). Call this asymmetry out so it isn't mistaken for a bug. Caveat: for very fast apps the earliest request can fire before capture attaches. If boot traffic looks missing, don't conclude "no request fired" — reload (`agent-view eval "location.reload()"`) and re-check before deciding.
288
-
289
- `[req=N]` handles are reallocated on every list call (like `dom` refs) — expand from the most recent list. Sensitive headers are redacted by default (`--raw-headers` reveals them). Response/request **bodies** require `"captureBody": true` in config; WebSocket frame payloads are visible by default.
290
-
291
- Standard pattern for "did the save call fire and succeed?":
292
- ```bash
293
- agent-view network --clear
294
- agent-view click --filter "Save"
295
- agent-view wait --filter "Saved"
296
- agent-view network --url "*/api/save*" # expect one POST with status 200
297
- ```
298
-
299
- ### Targets (`targets`)
300
-
301
- When `--window` doesn't show what you expected, or you need a worker target id for `eval`/`console`:
302
-
303
- ```bash
304
- agent-view targets # everything connectable
305
- agent-view targets --type shared_worker,service_worker # filter
306
- agent-view targets --json # machine-readable
307
- ```
308
-
309
- You usually don't need this — `eval --target <substring>` and `--window <name>` both do fuzzy matching. Reach for `targets` when the substring is ambiguous.
310
-
311
- `targets` prints ids truncated to 8 chars, and `--target` / `--window` accept that printed handle (case-insensitive id prefix, ≥4 chars) as well as a full id or a title/URL substring. An ambiguous prefix is reported as ambiguous rather than resolved to an arbitrary target. Worker targets often share a blank title, so the printed id prefix is the reliable handle for them.
312
-
313
- ### Scene / Canvas / WebGL (only when `webgl` is configured in agent-view.config.json)
314
-
315
- These commands read the scene graph from canvas-based rendering engines. Skip this section if the project has no `webgl` field in config.
316
-
317
- ```bash
318
- agent-view scene # Scene graph from configured engine
319
- agent-view scene --filter "player" # Filter by object name/type
320
- agent-view scene --verbose # Extended props (scale, alpha, rotation)
321
- agent-view scene --diff # Changes since last call
322
- agent-view scene --compact # Merge single-child chains onto one line (reduces output)
323
- agent-view snap # DOM + Scene combined
324
- agent-view snap --scale 0.5 # DOM + Scene + Screenshot (path appended as === Screenshot === section)
325
- ```
36
+ Every command takes `--window <id|name>`. Refs (`[ref=N]`) are session-scoped — after HMR or navigation, re-run `dom`
37
+ for fresh ones.
326
38
 
327
39
  ## Picking the right tool
328
40
 
329
41
  Verifications cost very different amounts. Pick the cheapest tool that can actually answer the question:
330
42
 
331
- | The question is about… | Use | Why |
332
- |---|---|---|
333
- | Element existence / text / role | `dom --filter` | Cheapest, structured, no vision tokens |
334
- | Count of matching elements | `dom --filter X --count` | Single integer, no tree output, no ref mutations |
335
- | App state, store contents, computed values | `eval "expr"` | DOM doesn't expose JS state; reading the tree to infer it is wasteful and unreliable |
336
- | Does `window.X` / a globally-exposed API exist? | `eval "typeof window.X"` | DOM doesn't show JS globals; only authoritative check |
337
- | An element that has not rendered yet | `wait --filter "<text>"` | Exits on appearance and non-zero on timeout — a real gate, unlike a fixed pause |
338
- | State *trajectory* — what changed during/after an action | `watch "expr" --until …` or `--max-changes 1` | `eval` shows the final snapshot only; `watch` shows the diffs in order |
339
- | Worker logic (SharedWorker / ServiceWorker) | `eval --target <name>` | Workers have no DOM at all |
340
- | Did the last action throw or warn? | `console --clear` before, `console --level error,warn` after | Catches errors that don't surface in the DOM |
341
- | What happened over a long / flaky / reload-spanning run | `logs start` … `logs tail --grep`/`--since` | Durable one-line-per-record timeline of page + workers; `console` loses it on idle shutdown |
342
- | Layout/visual of a specific element | `screenshot --crop "<element>"` | ~1.6k tokens (1 tile) — crops to bounding box, massive token win |
343
- | Layout, spacing, full-window visual regression | `screenshot --scale 0.5` | The only tool that sees pixels — but expensive (~6k tokens), use last |
344
- | Canvas/WebGL scene contents | `scene --diff` | DOM is empty for canvas apps |
345
- | What DOM nodes changed after an interaction | `dom --diff` | Returns only `+`/`-` lines; much cheaper than re-reading the full tree |
346
- | Selecting a file for an input that exists in the DOM | `upload --selector` | No picker opens at all; works on hidden inputs, which have no ref |
347
- | Selecting a file when the input appears only mid-click | `dialog arm --file` then `click` | The only way — the input does not exist before the click and is gone after |
348
- | The window stopped responding after a click | `dialog` | Shows whether a modal was answered, and what the app was told |
349
-
350
- When two tools could answer the same question, prefer the one higher up the table. A common mistake is screenshotting to check "is the count = 5?" when `eval "store.counter"` returns the number directly for ~50 tokens.
351
-
352
- ## Verification Workflow
353
-
354
- ### Execution discipline (read first, every run)
355
-
356
- A run can produce one of three outcomes per step: **pass**, **fail**, **requires_visual_review**. There is no fourth bucket called "actually fine, here's why". A failed `Expected:` line is FAIL.
43
+ | The question is about… | Use | Why |
44
+ |------------------------------------------------------------------|--------------------------------------------------------------|---------------------------------------------------------------------------------------------|
45
+ | Element existence / text / role | `dom --filter` | Cheapest, structured, no vision tokens |
46
+ | Count of matching elements | `dom --filter X --count` | Single integer, no tree output, no ref mutations |
47
+ | App state, store contents, computed values | `eval "expr"` | DOM doesn't expose JS state; reading the tree to infer it is wasteful and unreliable |
48
+ | Does `window.X` / a globally-exposed API exist? | `eval "typeof window.X"` | DOM doesn't show JS globals; only authoritative check |
49
+ | An element that has not rendered yet | `wait --filter "<text>"` | Exits on appearance and non-zero on timeout — a real gate, unlike a fixed pause |
50
+ | State *trajectory* — what changed during/after an action | `watch "expr" --until …` or `--max-changes 1` | `eval` shows the final snapshot only; `watch` shows the diffs in order |
51
+ | Worker logic (SharedWorker / ServiceWorker) | `eval --target <name>` | Workers have no DOM at all |
52
+ | Did the last action throw or warn? | `console --clear` before, `console --level error,warn` after | Catches errors that don't surface in the DOM |
53
+ | Did an expected API call fire, and with what status? | `network --clear` before, `network --url "<glob>"` after | Captures eagerly from launch; the DOM shows the result, not the call |
54
+ | What happened over a long / flaky / reload-spanning run | `logs start` … `logs tail --grep`/`--since` | Durable one-line-per-record timeline of page + workers; `console` loses it on idle shutdown |
55
+ | Layout/visual of a specific element | `screenshot --crop "<element>"` | ~1.6k tokens (1 tile) — crops to bounding box, massive token win |
56
+ | Layout, spacing, full-window visual regression | `screenshot --scale 0.5` | The only tool that sees pixels — but expensive (~6k tokens), use last |
57
+ | Canvas/WebGL scene contents | `scene --diff` | DOM is empty for canvas apps |
58
+ | What DOM nodes changed after an interaction | `dom --diff` | Returns only `+`/`-` lines; much cheaper than re-reading the full tree |
59
+ | Selecting a file for an input that exists in the DOM | `upload --selector` | No picker opens at all; works on hidden inputs, which have no ref |
60
+ | Selecting a file when the input appears only mid-click | `dialog arm --file` then `click` | The only way — the input does not exist before the click and is gone after |
61
+ | Does any user action reach this code? | `coverage --clear` before, `coverage --file X` after | The only tool that answers it; a diff cannot |
62
+ | What handler is bound to this element, and where is it declared? | `listeners --filter "<text>"` | Gives `file:line` without reading the source |
63
+ | The window stopped responding after a click | `dialog` | Shows whether a modal was answered, and what the app was told |
64
+
65
+ When two tools could answer the same question, prefer the one higher up the table.
66
+
67
+ ## Execution discipline (read first, every run)
68
+
69
+ A run produces one of three outcomes per step: **pass**, **fail**, **requires_visual_review**. There is no fourth bucket
70
+ called "actually fine, here's why". A failed `Expected:` line is FAIL.
357
71
 
358
72
  **How to run the commands themselves:**
359
73
 
360
- - **Never discard output.** No `>/dev/null`, no `2>&1` to nowhere, no `| head -1` on a command whose
361
- failure you have not yet read. Every agent-view command prints either the evidence or the reason
362
- it failed; a suppressed `click` that matched nothing looks exactly like a successful one.
74
+ - **Never discard output.** No `>/dev/null`, no `2>&1` to nowhere, no `| head -1` on a command whose failure you have
75
+ not yet read. Every agent-view command prints either the evidence or the reason it failed; a suppressed `click` that
76
+ matched nothing looks exactly like a successful one.
363
77
  - **Chain with `&&`, not newlines.** Newline-separated commands keep running after a failure, so a
364
78
  broken first step is followed by three steps acting on the wrong state. `&&` stops at the first
365
79
  non-zero exit.
366
80
  - **One action, then one check.** `click` is not evidence. The evidence is the `dom --diff`,
367
- `dom --filter … --count`, `eval`, or `console --level error` you run after it. A block of three
368
- clicks in a row with no check between them proves nothing about any of them.
369
- - **Call the `agent-view` binary.** Install it once in the target project
370
- (`pnpm add -D @petukhovart/agent-view`) and call `agent-view …` or `pnpm exec agent-view …`.
371
- Do not prefix every call with `npx <package>`: it re-resolves the package on each invocation and
372
- falls outside this skill's `allowed-tools`, so each call needs a fresh permission prompt.
373
-
374
- These heuristics catch real bugs. Skipping them is how a run silently passes while the bug sits in plain sight in the same data:
375
-
376
- 1. **A failed expectation is FAIL.** If output disagrees with what the step expected, mark `fail` and continue. Do not soften the expectation. Do not invent prose explanations inline ("label reuse", "convention", "arithmetic off"). Justifications belong in the bug report after the run, never in the per-step log.
377
-
378
- 2. **UI-vs-model mismatch is the bug, not noise.** When a count or hierarchy check returns `match: false`:
379
- - Default hypothesis: the UI renderer is wrong.
380
- - Before considering "the filter matched something extra in some side panel", query bounding boxes and ancestor chains of the matched elements. Two matches at the same x-coordinate in adjacent y rows = sibling rows in one list = renderer bug.
381
- - Do not dismiss DOM/model divergence with "scene-graph is the source of truth". The model is one representation; the bug may live in the gap between model and UI.
81
+ `dom --filter … --count`, `eval`, or `console --level error` you run after it. Three clicks in a row with no check
82
+ between them prove nothing about any of them.
83
+ - **Never sleep for a fixed time.** No `sleep`, no `timeout`, no `ping -n N 127.0.0.1` as a delay (agents reach for
84
+ `ping` when the harness blocks `sleep` — the same mistake in worse clothing). A fixed pause is either too short, and
85
+ you assert against a half-rendered UI, or too long, and you burn wall-clock on every step. There is a condition-based
86
+ wait for every kind of signal:
87
+
88
+ | You are waiting for… | Use |
89
+ |---|---|
90
+ | An element to render | `wait --filter "<text>"` |
91
+ | A store/state value | `watch "<expr>" --until "<expr>"` |
92
+ | A log line | `console --follow --until "<pattern>"` |
93
+ | A request to fire | `network --follow --until "<url>"` |
94
+
95
+ If none of these fits, poll `dom --filter X --count` with an explicit attempt cap and report how many attempts it
96
+ took.
97
+ - **Call the `agent-view` binary.** Install it once in the target project (`pnpm add -D @petukhovart/agent-view`, or
98
+ your package manager's equivalent) and call
99
+ `agent-view …` / `pnpm exec agent-view …`. Prefixing every call with `npx <package>` re-resolves the package on each
100
+ invocation, and in permission-gated harnesses each call then needs a fresh approval prompt.
101
+
102
+ These heuristics catch real bugs. Skipping them is how a run silently passes while the bug sits in plain sight in the
103
+ same data:
104
+
105
+ 1. **A failed expectation is FAIL.** If output disagrees with what the step expected, mark `fail` and continue. Do not
106
+ soften the expectation. Do not invent prose explanations inline ("label reuse",
107
+ "convention", "arithmetic off"). Justifications belong in the bug report after the run, never in the per-step log.
108
+
109
+ 2. **UI-vs-model mismatch is the bug, not noise.** When a count or hierarchy check returns
110
+ `match: false`, the default hypothesis is that the UI renderer is wrong. Before reaching for "the filter matched
111
+ something extra in a side panel", query the bounding boxes and ancestor chains of the matched elements — two matches
112
+ at the same x in adjacent y rows are sibling rows in one list, i.e. a renderer bug. The model is one representation,
113
+ not the source of truth; the bug may live in the gap between model and UI.
114
+
115
+ 3. **Defensive eval reads.** Sentinel-check every `node.field` read (`transform.x`,
116
+ `transform.width`) before using it in arithmetic. A renamed field silently returns `NaN`/`null`, which fail-passes
117
+ downstream comparisons. Add `isFinite(value)` / `value !== undefined` guards inline.
118
+
119
+ 4. **No hardcoded literal IDs.** A hardcoded node-ID prefix that no longer matches the current scene degrades the whole
120
+ check to a silent no-op. Verify at least one expected ID exists; if not, derive IDs by role at runtime, proceed with
121
+ the corrected lookup, and say the plan needs an ID refresh.
122
+
123
+ 5. **Reload checkpoint is not optional.** If the feature mutated persisted structure, run one:
124
+ `agent-view eval "location.reload()"`, wait for the app to come back, re-read the structural signature, diff. Drift
125
+ is a real bug, not a "fixed-up on save".
126
+
127
+ 6. **Invariants run first or fail closed.** When the plan states invariants, execute those steps before the
128
+ action-specific checks. A failed invariant is FAIL for that invariant *and* a flag on the rest of the run — keep
129
+ running the remaining steps, tagged "trust-impaired until invariant restored".
130
+
131
+ 7. **Never claim a `window.*` API is missing without `eval`.** Before reporting "API not exposed" /
132
+ "global X doesn't exist" / "the host doesn't expose Y", run `agent-view eval "typeof window.X"`
133
+ and report the literal result (`"undefined"` / `"object"` / `"function"`). DOM scraping cannot answer this — globals
134
+ are not in the AX tree. If it returns `"undefined"` the API really is absent from the main world; anything else means
135
+ the API is reachable and your earlier conclusion was wrong. No exceptions, no "I checked the source code instead".
382
136
 
383
- 3. **Defensive eval reads.** Every `node.field` read (e.g. `transform.x`, `transform.width`) must be sentinel-checked before being used in arithmetic. A renamed field silently returns `NaN`/`null`, which fail-passes downstream comparisons. Add `isFinite(value)` / `value !== undefined` guards inline.
384
-
385
- 4. **No hardcoded literal IDs.** A hardcoded node-ID prefix that no longer matches the current scene degrades the whole check to a silent no-op. Verify at least one expected ID exists; if not, derive IDs by role at runtime and proceed with the corrected lookup, and say the plan needs an ID refresh.
386
-
387
- 5. **Reload checkpoint is not optional.** If the feature mutated persisted structure, run one: `agent-view eval "location.reload()"`, wait for the app to come back, re-read the structural signature, diff. Drift is a real bug, not a "fixed-up on save".
388
-
389
- 6. **Invariants run first or fail closed.** When the plan states invariants, execute those steps before the action-specific checks. A failed invariant is FAIL for that invariant *and* a flag on the rest of the run — keep running the remaining steps, tag them as "trust-impaired until invariant restored".
137
+ ## Verification Workflow
390
138
 
391
- 7. **Never claim a `window.*` API is missing without `eval`.** Before reporting "API not exposed" / "global X doesn't exist" / "the host doesn't expose Y", you MUST run `agent-view eval "typeof window.X"` and report the literal result (`"undefined"` / `"object"` / `"function"`). DOM scraping cannot answer this question — globals are not in the AX tree. If `eval` returns `"undefined"`, the API really is absent from the main world; if it returns anything else, the API is reachable and your earlier conclusion was wrong. No exceptions, no "I checked the source code instead".
139
+ Run the whole thing inline — **no subagent**. Resolve the window id once with `agent-view discover`
140
+ if you need `--window`.
392
141
 
393
- ### Ad-hoc Mode (standalone)
142
+ ### Ad-hoc mode (standalone)
394
143
 
395
144
  After making code changes:
396
145
 
397
- 1. **Determine affected areas** from git diff
398
- 2. **Ensure app is running**: `agent-view launch` or `agent-view discover`
399
- 3. **Inspect DOM**: `agent-view dom --filter "<area>" --depth 2` — check structure matches expectations
400
- 4. **Interact if needed**: `agent-view click`/`fill` → `agent-view dom --filter` to verify state changed
401
- 5. **For canvas apps**: `agent-view scene --diff` to see what changed
402
- 6. **For non-DOM truth** (store, computed values, worker state): `agent-view eval` — much cheaper than reading the DOM tree to infer state
403
- 7. **After any interaction that could fail silently**: `agent-view console --level error` — catches uncaught exceptions, network failures, framework warnings
404
- 8. **Screenshot only for final visual confirm**: `agent-view screenshot --scale 0.5` — captures layout/styling that DOM can't reveal
405
-
406
- ### Scenario Execution Mode (from plan)
407
-
408
- When UI scenarios are pre-generated (e.g., from a plan file with `## UI Scenarios` section):
409
-
410
- 1. **Read scenario steps** with symbolic refs (`$var` notation)
411
- 2. **Resolve each $var**: `agent-view dom --filter "<text>" --depth 3` → map to ref ID
412
- 3. **Execute steps** sequentially: fill, click, dom --filter (verify expected outcome)
413
- 4. **Screenshot**: `agent-view screenshot --scale 0.5` — only on FAIL and at E2E scenario end, not every step
414
- 5. **Report per-scenario**: PASS / FAIL with reason and evidence
415
-
416
- This mode works with any workflow that generates plan files with UI scenarios.
146
+ 1. **Determine affected areas** from `git diff` — every changed file that renders or drives UI needs at least one check.
147
+ 2. **Ensure the app is running**: `agent-view launch` (or `agent-view discover`).
148
+ 3. **Inspect DOM**: `agent-view dom --filter "<area>" --depth 2` — structure matches expectations.
149
+ 4. **Interact**: `agent-view click`/`fill` → `agent-view dom --filter` to verify the state changed.
150
+ 5. **For canvas apps**: `agent-view scene --diff`.
151
+ 6. **For non-DOM truth** (store, computed values, worker state): `agent-view eval`.
152
+ 7. **Before claiming code is unreachable**: `agent-view coverage --clear` → the action →
153
+ `agent-view coverage --file "<file>"`. Reading the diff is not evidence either way; an empty result narrows the claim
154
+ to "this action does not reach it".
155
+ 8. **After any interaction that could fail silently**: `agent-view console --level error` — catches uncaught exceptions,
156
+ network failures, framework warnings.
157
+ 9. **Screenshot last, for visual confirm only**: `agent-view screenshot --scale 0.5`.
158
+
159
+ ### Scenario mode (from a plan)
160
+
161
+ When UI scenarios are pre-generated (e.g. a plan file with a `## UI Scenarios` section): read the steps, resolve each
162
+ symbolic `$var` via `agent-view dom --filter "<text>" --depth 3` to a ref, execute the steps in order, and verify each
163
+ expected outcome with `dom --filter`. Screenshot only on FAIL and at the end of an E2E scenario, never per step.
417
164
 
418
165
  ### Reporting
419
166
 
420
- Run the whole thing inline — **no subagent**. Resolve the window id once with `agent-view discover` if you need `--window`.
421
-
422
- Report a tight summary: passed / failed / visual-review counts, one line per failure, invariant violations called out separately. Don't paste raw stdout unless asked. After 2–3 consecutive failures, stop and distinguish "the plan is stale" (hardcoded IDs no longer match the current UI) from "the feature is broken" (invariants violated on a current scene) — they need opposite fixes.
423
-
424
- ### Design Conformance (inline)
167
+ One line per step, so the report stays machine-readable:
425
168
 
426
- When you are given `(label, screenshot command, expected reference path)` rows — from a plan, or from the developer directly — execute them yourself, no subagent.
169
+ ```
170
+ <step label> | pass | fail | requires_visual_review — <evidence command and its result>
171
+ ```
427
172
 
428
- For each row:
429
- 1. Run the screenshot command (capture the saved file path from stdout).
430
- 2. `Read` both the captured image and the `expected_path`. If `expected_path` doesn't exist or is unreadable, mark the pair `skipped (expected_missing)` and move on.
431
- 3. Compare visually for: layout (relative position, alignment), sizing, color (dominant color family), typography (weight/size broadly), content presence (anything missing or extra), decorations (borders, shadows, dashed/solid lines, icons).
432
- 4. Report each pair as `match` / `minor_mismatch` / `major_mismatch` with a one-sentence deviation. Major = missing/wrong component, broken layout, wrong color family, wrong text content. Minor = <10px spacing drift, slight color shade, small decoration difference.
173
+ Close with passed / failed / visual-review counts, and call out invariant violations separately. Do not paste raw stdout
174
+ unless asked.
433
175
 
434
- Tolerance default: a designer's code-review level — flag what they'd notice, ignore anti-aliasing noise. Don't speculate about CSS causes — describe what looks different and let the parent / user decide.
176
+ **Design conformance** — when you are handed `(label, screenshot command, expected reference path)`
177
+ rows, follow [`references/design-conformance.md`](references/design-conformance.md).
435
178
 
436
179
  ## Resilience
437
180
 
438
- - **Stale refs:** After HMR, navigation, or state change — re-run `dom` for fresh refs before interacting
439
- - **Element not found:** `agent-view wait --filter "<text>" --timeout 5` (covers the render delay after HMR). If it times out — report FAIL. Do not insert a blind pause and retry.
440
- - **CDP disconnect:** Run `agent-view discover` to check. If no windows — `agent-view launch` (auto-starts Electron/Browser/Tauri). On `PORT_CONFLICT` — surface PID/process and ask user.
441
- - **`CDP_TIMEOUT` error:** the command hit the server-side deadline; cached CDP sessions for that port were dropped, so retry once. Repeated timeouts mean the app's DevTools endpoint is wedged — restart the app.
442
- - **Max retries per command:** 2. After that — SKIP scenario step with warning
443
-
444
- ## Important Notes
445
-
446
- - **Refs are session-scoped** — after HMR or navigation, run `dom` again for fresh refs
447
- - **Multiple windows**: use `--window <id>` from `discover` output when titles overlap
448
- - **Multiwindow**: all commands support `--window` flag
449
- - **Output format**: plain text (DOM, scene), JSON (discover only), file path (screenshot)
450
- - **Lazy server**: auto-starts on first call, shuts down after 5min idle
181
+ - **Element not found:** `agent-view wait --filter "<text>" --timeout 5` covers the render delay after HMR. If it times
182
+ out — report FAIL.
183
+ - **Stale refs:** re-run `dom` after HMR, navigation, or a state change before interacting again.
184
+ - **CDP disconnect:** `agent-view discover` to check. If no windows — `agent-view launch`. On
185
+ `PORT_CONFLICT` the CLI reports the owning PID and process name; surface it and ask the user to free the port. Never
186
+ kill a foreign process from this skill.
187
+ - **`CDP_TIMEOUT`:** the command hit the server-side deadline and cached sessions for that port were dropped, so retry
188
+ once. Repeated timeouts mean the app's DevTools endpoint is wedged — restart it.
189
+ - **Retry budget: 2 per command**, then SKIP the step with a warning. Two covers a transient CDP drop; a third repeat
190
+ means the app or the plan is wrong, not the call. After two or three consecutive failures, stop and distinguish "the
191
+ plan is stale" (hardcoded IDs no longer match the UI) from "the feature is broken" (invariants violated on a current
192
+ scene) — they need opposite fixes.
451
193
 
452
194
  ## Token Optimization
453
195
 
454
- Vision tokens dominate cost. One full-res screenshot ≈ 19k tokens (1920×1080, 12 tiles).
455
-
456
- | Technique | Savings |
457
- |---|---|
458
- | `agent-view eval "expr"` for state checks | Returns one value (~50 tokens) instead of a DOM/screenshot |
459
- | `agent-view dom --filter "row" --count` | Single integer answer — zero tree tokens |
460
- | `agent-view dom --filter X --depth 2` | Narrow tree to relevant subtree, cap depth |
461
- | `agent-view screenshot --scale 0.5` | ~3× fewer vision tokens (4 tiles) |
462
- | `agent-view screenshot --scale 0.25` | ~12× fewer vision tokens (1 tile, ~1.6k tokens) |
463
- | `agent-view screenshot --crop "<element>"` | ~12× fewer in best case (1 tile) — crops to element bounding box |
464
- | DOM/eval-first: screenshot only for final visual confirm | Eliminates most screenshot calls |
196
+ Vision tokens dominate cost: a full-res screenshot is ≈19k tokens (1920×1080, 12 tiles), `--scale 0.5`
197
+ ≈6k (4 tiles), `--scale 0.25` and `--crop` ≈1.6k (1 tile). A text answer is ~50. So `--depth` and
198
+ `--filter` on `dom`, and `--count` where a number is the whole answer, cost near nothing by comparison.
465
199
 
466
- **Default rule**: if the answer is a value → `eval`; if the answer is "is element X visible/correct?" → `agent-view dom --filter`; if you need pixels for a specific section → `screenshot --crop "<element>"` (one tile); only call `screenshot --scale 0.5` for full-window visual proof.
200
+ **Default rule**: the answer is a value → `eval`; the answer is "is element X visible/correct?" →
201
+ `dom --filter`; you need pixels for one section → `screenshot --crop "<element>"`; only call
202
+ `screenshot --scale 0.5` for full-window visual proof.