@petukhovart/agent-view 0.13.1 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +21 -21
- package/LICENSE +21 -21
- package/README.md +51 -0
- package/dist/cdp/transport.d.ts.map +1 -1
- package/dist/cdp/transport.js +139 -15
- package/dist/cdp/transport.js.map +1 -1
- package/dist/cdp/types.d.ts +51 -0
- package/dist/cdp/types.d.ts.map +1 -1
- package/dist/cdp/types.js.map +1 -1
- package/dist/cli/commands/click.d.ts +1 -0
- package/dist/cli/commands/click.d.ts.map +1 -1
- package/dist/cli/commands/click.js +2 -0
- package/dist/cli/commands/click.js.map +1 -1
- package/dist/cli/commands/coverage.d.ts +14 -0
- package/dist/cli/commands/coverage.d.ts.map +1 -0
- package/dist/cli/commands/coverage.js +34 -0
- package/dist/cli/commands/coverage.js.map +1 -0
- package/dist/cli/commands/listeners.d.ts +11 -0
- package/dist/cli/commands/listeners.d.ts.map +1 -0
- package/dist/cli/commands/listeners.js +34 -0
- package/dist/cli/commands/listeners.js.map +1 -0
- package/dist/cli/index.js +30 -0
- package/dist/cli/index.js.map +1 -1
- package/dist/inspectors/coverage/index.d.ts +17 -0
- package/dist/inspectors/coverage/index.d.ts.map +1 -0
- package/dist/inspectors/coverage/index.js +65 -0
- package/dist/inspectors/coverage/index.js.map +1 -0
- package/dist/inspectors/listeners/index.d.ts +4 -0
- package/dist/inspectors/listeners/index.d.ts.map +1 -0
- package/dist/inspectors/listeners/index.js +30 -0
- package/dist/inspectors/listeners/index.js.map +1 -0
- package/dist/server/server.d.ts +9 -0
- package/dist/server/server.d.ts.map +1 -1
- package/dist/server/server.js +70 -2
- package/dist/server/server.js.map +1 -1
- package/package.json +1 -1
- package/skills/verify/SKILL.md +157 -423
- package/skills/verify/references/commands.md +358 -0
- package/skills/verify/references/design-conformance.md +23 -0
- package/dist/cdp/console-stream.d.ts +0 -35
- package/dist/cdp/console-stream.d.ts.map +0 -1
- package/dist/cdp/console-stream.js +0 -103
- package/dist/cdp/console-stream.js.map +0 -1
- package/dist/inspectors/dom.d.ts +0 -17
- package/dist/inspectors/dom.d.ts.map +0 -1
- package/dist/inspectors/dom.js +0 -131
- package/dist/inspectors/dom.js.map +0 -1
- package/dist/inspectors/scene/pixi.d.ts +0 -3
- package/dist/inspectors/scene/pixi.d.ts.map +0 -1
- package/dist/inspectors/scene/pixi.js +0 -52
- package/dist/inspectors/scene/pixi.js.map +0 -1
package/skills/verify/SKILL.md
CHANGED
|
@@ -1,468 +1,202 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: verify
|
|
3
|
-
description: "Visual
|
|
3
|
+
description: "Visual and runtime verification of a running app over CDP. Use when user looking for: checking a UI change, reproducing a visual bug, driving interactions, or when any workflow phase must inspect the live app — DOM, screenshots, store and worker state, console errors, network calls, or which code an action actually reached, or mention: verify, agent-view, check UI, visual regression"
|
|
4
4
|
allowed-tools: Bash(agent-view *), Read
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
# Visual Verification with agent-view
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
`agent-view` inspects and drives a running desktop app over Chrome DevTools Protocol.
|
|
10
10
|
|
|
11
11
|
## Prerequisites
|
|
12
12
|
|
|
13
|
-
The target project
|
|
14
|
-
|
|
15
|
-
|
|
13
|
+
The target project needs `agent-view.config.json` in its root (`agent-view init` writes one) and CDP enabled in the
|
|
14
|
+
app — e.g. `--remote-debugging-port=9876` for Electron. Avoid `9222`: it is Chrome's own default and collides when
|
|
15
|
+
Chrome is open. The server is lazy: it starts on the first call and shuts down after 5 min idle.
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
## Commands
|
|
18
18
|
|
|
19
|
-
|
|
19
|
+
Flags, output contracts and per-command failure modes live in **[`references/commands.md`](references/commands.md)** —
|
|
20
|
+
read it before your first call in a session.
|
|
20
21
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
agent-view dom # DOM accessibility tree (default window)
|
|
35
|
-
agent-view dom --window <id|name> # Specific window
|
|
36
|
-
agent-view dom --filter "button" # Filter by text/role
|
|
37
|
-
agent-view dom --depth 3 # Limit tree depth
|
|
38
|
-
agent-view dom --compact # Merge single-child chains onto one line (~40-60% fewer tokens)
|
|
39
|
-
agent-view dom --count # Count of all visible nodes (single integer line)
|
|
40
|
-
agent-view dom --filter "row" --count # Count matching nodes — e.g. "does this table have 5 rows?"
|
|
41
|
-
agent-view dom --max-lines 200 # Hard line budget; refs for truncated nodes still stored
|
|
42
|
-
agent-view dom --diff # Lines changed since last dom call (+ added / - removed)
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
`--count` skips tree output and ref mutations — cheapest way to assert "element exists N times" without loading the full tree into context.
|
|
46
|
-
|
|
47
|
-
### Interaction
|
|
48
|
-
```bash
|
|
49
|
-
agent-view click <ref> # Click element by ref from dom output
|
|
50
|
-
agent-view click --filter "Save" # Find element by text and click
|
|
51
|
-
agent-view click --pos 100,200 # Click by coordinates — CANVAS ONLY, see below
|
|
52
|
-
agent-view click <ref> --double # Double-click (fires dblclick handlers); works with --filter / --pos too
|
|
53
|
-
agent-view fill <ref> "text" # Type into input field
|
|
54
|
-
agent-view drag --from <ref> --to <ref> # Drag element to another element by ref
|
|
55
|
-
agent-view drag --from-pos 50,80 --to-pos 200,300 # Drag by coordinates (for canvas / Pixi)
|
|
56
|
-
agent-view drag --from <ref> --to <ref> --steps 25 --hold-ms 60 # Smoother movement, longer hold
|
|
57
|
-
```
|
|
58
|
-
|
|
59
|
-
`drag` dispatches `mousePressed` → N × `mouseMoved` → `mouseReleased` via CDP. Endpoints can mix
|
|
60
|
-
ref and coordinate (e.g. `--from <ref> --to-pos 400,300`). For canvas/Pixi targets always use
|
|
61
|
-
`--from-pos`/`--to-pos` — derive the centroid via `agent-view eval` from the scene graph.
|
|
62
|
-
Refs are resolved fresh on each call, so window resizes between snapshots are tolerated.
|
|
63
|
-
Increase `--steps` for handlers using `globalpointermove` so intermediate frames are not skipped.
|
|
64
|
-
|
|
65
|
-
**Coordinates are a last resort.** `--pos` / `--from-pos` / `--to-pos` exist for canvas and WebGL,
|
|
66
|
-
where no ref exists. On DOM the correct order is `dom --filter "<text>"` → take the `[ref=N]` →
|
|
67
|
-
`click <ref>`, or `click --filter "<text>"` in one step. A coordinate pair breaks on any layout
|
|
68
|
-
shift, scroll, zoom, or window resize, and it clicks whatever now sits at that point — silently.
|
|
69
|
-
If you reach for `--pos` on a DOM element, first say why the ref was not usable.
|
|
70
|
-
|
|
71
|
-
### Modals & file pickers (`dialog`, `upload`)
|
|
72
|
-
|
|
73
|
-
A modal that agent-view cannot answer stops a run dead: the window looks frozen, every
|
|
74
|
-
later command times out, and nothing says why. Two kinds, handled differently.
|
|
75
|
-
|
|
76
|
-
**JS modals — `alert` / `confirm` / `prompt` / `beforeunload` — are answered for you.**
|
|
77
|
-
Nothing to set up. While agent-view is attached these never block the page: the default
|
|
78
|
-
standing answer is *dismiss*, and each one is recorded. `agent-view dialog` is the
|
|
79
|
-
reliable place to read that record; the console feed carries the same line
|
|
80
|
-
(`[agent-view] confirm auto-dismissed: <message>`) but only from the moment console
|
|
81
|
-
attaches, so a modal answered before your first `console` call reaches `dialog` alone.
|
|
82
|
-
|
|
83
|
-
```bash
|
|
84
|
-
agent-view dialog # standing answer + every modal this window has seen
|
|
85
|
-
agent-view dialog policy accept # confirm() → true from now on
|
|
86
|
-
agent-view dialog policy accept --text "name" # prompt() → "name"
|
|
87
|
-
agent-view dialog policy dismiss # back to the default
|
|
88
|
-
agent-view dialog dismiss # answer one that is open right now
|
|
89
|
-
agent-view dialog accept --text "x" # …ditto, accepting
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
`dialog accept` / `dialog dismiss` exist for a modal that was already open **before**
|
|
93
|
-
agent-view attached — that one produced no event, so no policy applied to it.
|
|
94
|
-
|
|
95
|
-
**Native file pickers never open — you answer them in advance.** Which mechanism applies
|
|
96
|
-
depends on how the app opens the picker, and `dialog arm` sets up all of them at once,
|
|
97
|
-
so you do not have to know:
|
|
98
|
-
|
|
99
|
-
```bash
|
|
100
|
-
# The input already exists in the DOM (even hidden) — no picker at all, cheapest path
|
|
101
|
-
agent-view upload --selector "#file-input" --file ./fixtures/a.png
|
|
102
|
-
agent-view upload --selector "#imgs" --file ./a.png --file ./b.png # multi-select
|
|
103
|
-
agent-view upload --ref 12 --file ./a.png # if the AX tree exposes it
|
|
104
|
-
|
|
105
|
-
# The input is created inside the click handler, or the app calls a native dialog API
|
|
106
|
-
agent-view dialog arm --file ./fixtures/a.png # then click the button that opens it
|
|
107
|
-
agent-view dialog arm --cancel # act as if the user pressed Cancel
|
|
108
|
-
agent-view dialog disarm # let real pickers open again
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
Rules that actually bite:
|
|
112
|
-
|
|
113
|
-
- **Arm before the click.** A picker cannot be caught once it is open. `arm` is one-shot —
|
|
114
|
-
it is spent by the first picker and interception turns itself off, so a later click
|
|
115
|
-
opens a real OS dialog.
|
|
116
|
-
- **Click through `agent-view click`, never `eval "el.click()"`.** Chromium refuses to
|
|
117
|
-
open a file picker without user activation, and an eval-driven click carries none: the
|
|
118
|
-
picker is silently dropped and never intercepted.
|
|
119
|
-
- **Hidden inputs have no ref.** `display:none` / `v-show="false"` keeps them out of the
|
|
120
|
-
AX tree, so `dom` never prints one. Use `--selector`. `upload` has no `--filter` on
|
|
121
|
-
purpose: an accessible name lands on the label, not on the input behind it.
|
|
122
|
-
- **`beforeunload` is always dismissed**, whatever the policy — accepting it navigates
|
|
123
|
-
away and loses the state you are checking.
|
|
124
|
-
- Paths are resolved against your cwd and must exist — CDP accepts a bad path silently and
|
|
125
|
-
the app then reads an empty file.
|
|
126
|
-
- `agent-view dialog` after the fact shows what was intercepted and what was answered.
|
|
127
|
-
|
|
128
|
-
Known limits: `showOpenFilePicker()` (File System Access API) exposes no input to fill, so
|
|
129
|
-
it can only be cancelled. Electron does not implement `window.prompt` at all. Native
|
|
130
|
-
dialogs opened straight from an Electron **main** process (`dialog.showOpenDialog` behind
|
|
131
|
-
an IPC channel) are out of reach — CDP does not see the main process.
|
|
132
|
-
|
|
133
|
-
### Waiting (`wait`)
|
|
134
|
-
|
|
135
|
-
```bash
|
|
136
|
-
agent-view wait --filter "Saved" # until the text appears in the AX tree
|
|
137
|
-
agent-view wait --filter "Saved" --timeout 20 # max wait in seconds (default 10)
|
|
138
|
-
agent-view wait --filter "Row 5" --window "Main" # specific window
|
|
139
|
-
```
|
|
140
|
-
|
|
141
|
-
Exits as soon as the element appears; exits non-zero on timeout — so `&&` after it is a real gate.
|
|
142
|
-
|
|
143
|
-
**Never sleep for a fixed time.** No `sleep`, no `timeout`, no `ping -n N 127.0.0.1` as a delay
|
|
144
|
-
(agents reach for `ping` when the harness blocks `sleep` — it is the same mistake in worse
|
|
145
|
-
clothing). A fixed pause is either too short, and you assert against a half-rendered UI, or too
|
|
146
|
-
long, and you burn wall-clock on every step. agent-view has a condition-based wait for every kind
|
|
147
|
-
of signal:
|
|
148
|
-
|
|
149
|
-
| You are waiting for… | Use |
|
|
150
|
-
|---|---|
|
|
151
|
-
| An element to render | `wait --filter "<text>"` |
|
|
152
|
-
| A store/state value | `watch "<expr>" --until "<expr>"` |
|
|
153
|
-
| A log line | `console --follow --until "<pattern>"` |
|
|
154
|
-
| A request to fire | `network --follow --until "<url>"` |
|
|
155
|
-
|
|
156
|
-
If none of these fits, poll: `dom --filter X --count` in a loop with an explicit attempt cap, and
|
|
157
|
-
report how many attempts it took.
|
|
158
|
-
|
|
159
|
-
### Screenshots
|
|
160
|
-
```bash
|
|
161
|
-
agent-view screenshot --scale 0.5 # Recommended: JPEG at half-res (~3× fewer vision tokens)
|
|
162
|
-
agent-view screenshot --scale 0.5 --window <id> # Specific window
|
|
163
|
-
agent-view screenshot --crop "Sidebar" # Crop to element bounding box (~1.6k tokens — 12× win)
|
|
164
|
-
agent-view screenshot --crop "Chart" --scale 0.5 # Crop + scale (stacks)
|
|
165
|
-
agent-view screenshot --crop "Active bookings" --crop-up 1 # Crop the card, not just its heading
|
|
166
|
-
agent-view screenshot # Full-res PNG (expensive: ~19k tokens at 1920×1080)
|
|
167
|
-
```
|
|
168
|
-
|
|
169
|
-
`--crop <filter>` resolves the element with the same filter syntax as `dom --filter`, then crops the screenshot to its bounding box. Prefer `--crop` over full-window screenshots whenever you only need to inspect a specific section. Falls back to full-window with a stderr warning if the filter matches nothing.
|
|
170
|
-
|
|
171
|
-
A text filter usually matches the text-bearing node, so cropping on a section title returns a thin strip of that title. `--crop-up <n>` climbs `n` element ancestors before cropping — use `1` (sometimes `2`) to get the surrounding card/section. When a crop comes back text-sized, the command says so on stderr.
|
|
172
|
-
|
|
173
|
-
### Runtime State (`eval`)
|
|
174
|
-
|
|
175
|
-
Reads runtime values DOM/screenshot can't reveal — store contents, computed flags, worker internals.
|
|
176
|
-
**Requires `"allowEval": true` in `agent-view.config.json`** — if the call returns "eval is disabled", tell the user to add the flag rather than working around it.
|
|
177
|
-
|
|
178
|
-
```bash
|
|
179
|
-
agent-view eval "store.state.user.role" # default page target
|
|
180
|
-
agent-view eval --window "Settings" "router.currentRoute.path"
|
|
181
|
-
agent-view eval --target sync-worker "self.queue.length" # SharedWorker / ServiceWorker by id or substring
|
|
182
|
-
agent-view eval --await "fetch('/api/health').then(r => r.status)"
|
|
183
|
-
agent-view eval --json "({ buttons: document.querySelectorAll('button').length })"
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
When to reach for `eval` instead of `dom`:
|
|
187
|
-
- The truth lives in JS state, not the DOM (Pinia/Vuex/Redux/Zustand store, Vue refs, computed values, app singletons).
|
|
188
|
-
- The target is a worker (`shared_worker`, `service_worker`, `worker`) — DOM doesn't exist there.
|
|
189
|
-
- You need a precise number/string answer, not a tree to scan.
|
|
190
|
-
- Verifying a `window.*` API or globally-exposed object exists. `eval` runs in the page's **main world**, so anything set on `window` directly or exposed via `contextBridge.exposeInMainWorld` is reachable. APIs placed only in an isolated-world preload (without `contextBridge`) will NOT be visible — that is not an agent bug, that is the host app's wiring.
|
|
191
|
-
|
|
192
|
-
### Reactive State (`watch`)
|
|
193
|
-
|
|
194
|
-
Streams JSON-patch diffs of an expression over time. Use when you need to see *what changed* between an action and a final state — `eval` shows the snapshot, `watch` shows the trajectory. **Requires `"allowEval": true`.**
|
|
195
|
-
|
|
196
|
-
```bash
|
|
197
|
-
agent-view watch "store.cart.total" # 250ms poll, default 10 changes or 30s
|
|
198
|
-
agent-view watch "appState" --until "appState.status === 'ready'" # wait-for-condition with diff log
|
|
199
|
-
agent-view watch "store.user" --max-changes 1 # capture exactly one change after a click
|
|
200
|
-
agent-view watch "appState" --json # NDJSON, machine-readable
|
|
201
|
-
```
|
|
202
|
-
|
|
203
|
-
When to reach for `watch` instead of `eval`:
|
|
204
|
-
- Debugging "the click did X but state shows Y — what happened in between?"
|
|
205
|
-
- Time-based assertions ("wait until store.status === 'ready'") — `--until` exits cleanly when truthy.
|
|
206
|
-
- Confirming an action triggered the *expected* sequence of mutations, not just the final state.
|
|
207
|
-
|
|
208
|
-
Output: `init` line (baseline), one line per RFC 6902 op (`replace /path old → new`, `add /items/0 ...`), final `stop` line with reason. Snapshot size cap 256 KB — narrow the expression (`store.x.y`, not `store`) for large objects.
|
|
209
|
-
|
|
210
|
-
### Console (`console`)
|
|
211
|
-
|
|
212
|
-
Streams `Runtime.consoleAPICalled` + `Log.entryAdded`. Use to confirm a flow finished without errors, or to surface a specific warning after an interaction.
|
|
213
|
-
|
|
214
|
-
```bash
|
|
215
|
-
agent-view console # buffered messages from auto-attached targets
|
|
216
|
-
agent-view console --level error,warn # filter
|
|
217
|
-
agent-view console --target sync-worker # one target (title/URL substring, same fuzzy semantics as eval --target)
|
|
218
|
-
agent-view console --target IJ56KL # one target (exact id)
|
|
219
|
-
agent-view console --follow --timeout 10 # stream window (use sparingly — 10s of waiting)
|
|
220
|
-
agent-view console --follow --until "ready" # exit as soon as a message contains "ready"
|
|
221
|
-
agent-view console --follow --until "/error/i" # exit on regex match (case-insensitive)
|
|
222
|
-
agent-view console --clear # baseline before an interaction
|
|
223
|
-
agent-view console --since "2026-04-26T10:00:00Z"
|
|
224
|
-
```
|
|
225
|
-
|
|
226
|
-
`--until` requires `--follow`. Exits immediately when a message matches (substring or `/regex/flags`). On timeout without match exits non-zero.
|
|
227
|
-
|
|
228
|
-
Standard pattern for "did this action error?":
|
|
229
|
-
```bash
|
|
230
|
-
agent-view console --clear
|
|
231
|
-
agent-view click --filter "Save"
|
|
232
|
-
agent-view wait --filter "Saved"
|
|
233
|
-
agent-view console --level error # expect "(no console messages)"
|
|
234
|
-
```
|
|
235
|
-
|
|
236
|
-
Default attached target types: `page`, `shared_worker`, `service_worker` (override via `consoleTargets` in config).
|
|
237
|
-
|
|
238
|
-
### Log feed (`logs`)
|
|
239
|
-
|
|
240
|
-
`console` reads a ring buffer that dies with the server; `logs` records the same messages — page *and* every worker — into one file you can grep later. Reach for it when a bug needs a timeline instead of a snapshot: intermittent failures, long scenarios, anything spanning reloads or worker restarts.
|
|
241
|
-
|
|
242
|
-
Feeds are scoped to the CDP port (since 0.13.1), so parallel worktrees each record their own app; give each one its own feed path (the default relative `.agent-view/console.log` already does, one per checkout).
|
|
243
|
-
|
|
244
|
-
```bash
|
|
245
|
-
agent-view logs start --truncate # start clean; keeps recording across reloads
|
|
246
|
-
agent-view logs # tail last 200 records (alias of `logs tail`)
|
|
247
|
-
agent-view logs tail --grep "ws closed" # substring or /regex/
|
|
248
|
-
agent-view logs tail --since -2m # -30s | -5m | -2h | 09:31 | 09:31:02.500 | ISO
|
|
249
|
-
agent-view logs tail --level error,warn -n 50
|
|
250
|
-
agent-view logs clear # truncate feed + drop console buffer (baseline)
|
|
251
|
-
agent-view logs status # attached targets, feed size, tick count
|
|
252
|
-
agent-view logs stop
|
|
253
|
-
```
|
|
254
|
-
|
|
255
|
-
Feed format — one record per line, always `HH:MM:SS.mmm [level] [type:id8] text`, local time, embedded newlines escaped as `\n`. That makes external `grep`/`awk` safe on it; a wrapped stack or JSON payload never breaks line-oriented filtering.
|
|
256
|
-
|
|
257
|
-
Default file `.agent-view/console.log` in the project root (override with `logFile` in config or `--file`). Caps at 8 MB, then rotates once to `<file>.prev` (`logMaxBytes` to change). Recording suspends the server's 5-min idle shutdown, so a long scenario keeps writing.
|
|
22
|
+
| Commands | What they cover |
|
|
23
|
+
|--------------------------------------|----------------------------------------------------------------|
|
|
24
|
+
| `launch` `discover` `stop` `targets` | start the app; list windows and worker targets |
|
|
25
|
+
| `dom` | accessibility tree — filter, count, diff, depth cap |
|
|
26
|
+
| `click` `fill` `drag` | interaction by ref, by text, or by coordinate |
|
|
27
|
+
| `wait` | block until an element appears; non-zero on timeout |
|
|
28
|
+
| `dialog` `upload` | JS modals and native file pickers |
|
|
29
|
+
| `screenshot` | full window, scaled, or cropped to one element |
|
|
30
|
+
| `eval` `watch` | state now / state trajectory over time (both need `allowEval`) |
|
|
31
|
+
| `console` `logs` | message ring buffer / durable file feed that survives reloads |
|
|
32
|
+
| `network` | requests, headers, timing, bodies, WebSocket frames |
|
|
33
|
+
| `coverage` `listeners` | which functions ran; what handler is bound to a node |
|
|
34
|
+
| `scene` `snap` | canvas / WebGL scene graph (only when `webgl` is configured) |
|
|
258
35
|
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
agent-view logs start --truncate
|
|
262
|
-
# … drive the scenario, reload, retry as many times as needed …
|
|
263
|
-
agent-view logs tail --level error,warn
|
|
264
|
-
agent-view logs tail --grep "/socket|retry/" --since -5m
|
|
265
|
-
agent-view logs stop
|
|
266
|
-
```
|
|
267
|
-
|
|
268
|
-
**Probes** (`--probe <file.js>[@target]`, requires `allowEval`) inject JS that logs into the same feed — use it when the evidence you need isn't logged by the app (wrap a method, count events, dump a scheduler). The probe is re-injected automatically whenever its context is gone: page reload, worker restart. Write it idempotent and let it report via plain `console.log`.
|
|
269
|
-
|
|
270
|
-
```bash
|
|
271
|
-
agent-view logs start --probe ./probes/orchestrator.js@shared_worker --probe ./probes/audio.js@index.html
|
|
272
|
-
```
|
|
273
|
-
|
|
274
|
-
### Network (`network`)
|
|
275
|
-
|
|
276
|
-
Request/response timeline, headers, timing, bodies, and WebSocket/SSE frames. Use to confirm an expected API call fired, diagnose a silent 404 / CORS block / missing auth header, or verify "button disabled until the network completes".
|
|
277
|
-
|
|
278
|
-
```bash
|
|
279
|
-
agent-view network # recent requests, newest at the bottom
|
|
280
|
-
agent-view network --req 3 # expand one: headers, timing, body / WS frame log
|
|
281
|
-
agent-view network --status 4xx,5xx # only failures (class or exact code, e.g. 404)
|
|
282
|
-
agent-view network --method POST # mutations among reads
|
|
283
|
-
agent-view network --type xhr,fetch # drop document/image/font noise
|
|
284
|
-
agent-view network --url "*/api/save*" # URL substring or * glob
|
|
285
|
-
agent-view network --follow --until "/api/save" # stream until a matching request fires
|
|
286
|
-
agent-view network --clear # baseline before an interaction
|
|
287
|
-
```
|
|
288
|
-
|
|
289
|
-
**Eager, unlike `console`.** `network` captures from app launch, so page-load traffic (initial XHR/fetch, auth handshakes, boot 404s) is usually already buffered by the time you call it — in most cases you don't need to reload. `console` is the opposite (lazy: attaches on first call, loses earlier output). Call this asymmetry out so it isn't mistaken for a bug. Caveat: for very fast apps the earliest request can fire before capture attaches. If boot traffic looks missing, don't conclude "no request fired" — reload (`agent-view eval "location.reload()"`) and re-check before deciding.
|
|
290
|
-
|
|
291
|
-
`[req=N]` handles are reallocated on every list call (like `dom` refs) — expand from the most recent list. Sensitive headers are redacted by default (`--raw-headers` reveals them). Response/request **bodies** require `"captureBody": true` in config; WebSocket frame payloads are visible by default.
|
|
292
|
-
|
|
293
|
-
Standard pattern for "did the save call fire and succeed?":
|
|
294
|
-
```bash
|
|
295
|
-
agent-view network --clear
|
|
296
|
-
agent-view click --filter "Save"
|
|
297
|
-
agent-view wait --filter "Saved"
|
|
298
|
-
agent-view network --url "*/api/save*" # expect one POST with status 200
|
|
299
|
-
```
|
|
300
|
-
|
|
301
|
-
### Targets (`targets`)
|
|
302
|
-
|
|
303
|
-
When `--window` doesn't show what you expected, or you need a worker target id for `eval`/`console`:
|
|
304
|
-
|
|
305
|
-
```bash
|
|
306
|
-
agent-view targets # everything connectable
|
|
307
|
-
agent-view targets --type shared_worker,service_worker # filter
|
|
308
|
-
agent-view targets --json # machine-readable
|
|
309
|
-
```
|
|
310
|
-
|
|
311
|
-
You usually don't need this — `eval --target <substring>` and `--window <name>` both do fuzzy matching. Reach for `targets` when the substring is ambiguous.
|
|
312
|
-
|
|
313
|
-
`targets` prints ids truncated to 8 chars, and `--target` / `--window` accept that printed handle (case-insensitive id prefix, ≥4 chars) as well as a full id or a title/URL substring. An ambiguous prefix is reported as ambiguous rather than resolved to an arbitrary target. Worker targets often share a blank title, so the printed id prefix is the reliable handle for them.
|
|
314
|
-
|
|
315
|
-
### Scene / Canvas / WebGL (only when `webgl` is configured in agent-view.config.json)
|
|
316
|
-
|
|
317
|
-
These commands read the scene graph from canvas-based rendering engines. Skip this section if the project has no `webgl` field in config.
|
|
318
|
-
|
|
319
|
-
```bash
|
|
320
|
-
agent-view scene # Scene graph from configured engine
|
|
321
|
-
agent-view scene --filter "player" # Filter by object name/type
|
|
322
|
-
agent-view scene --verbose # Extended props (scale, alpha, rotation)
|
|
323
|
-
agent-view scene --diff # Changes since last call
|
|
324
|
-
agent-view scene --compact # Merge single-child chains onto one line (reduces output)
|
|
325
|
-
agent-view snap # DOM + Scene combined
|
|
326
|
-
agent-view snap --scale 0.5 # DOM + Scene + Screenshot (path appended as === Screenshot === section)
|
|
327
|
-
```
|
|
36
|
+
Every command takes `--window <id|name>`. Refs (`[ref=N]`) are session-scoped — after HMR or navigation, re-run `dom`
|
|
37
|
+
for fresh ones.
|
|
328
38
|
|
|
329
39
|
## Picking the right tool
|
|
330
40
|
|
|
331
41
|
Verifications cost very different amounts. Pick the cheapest tool that can actually answer the question:
|
|
332
42
|
|
|
333
|
-
| The question is about…
|
|
334
|
-
|
|
335
|
-
| Element existence / text / role
|
|
336
|
-
| Count of matching elements
|
|
337
|
-
| App state, store contents, computed values
|
|
338
|
-
| Does `window.X` / a globally-exposed API exist?
|
|
339
|
-
| An element that has not rendered yet
|
|
340
|
-
| State *trajectory* — what changed during/after an action
|
|
341
|
-
| Worker logic (SharedWorker / ServiceWorker)
|
|
342
|
-
| Did the last action throw or warn?
|
|
343
|
-
|
|
|
344
|
-
|
|
|
345
|
-
| Layout
|
|
346
|
-
|
|
|
347
|
-
|
|
|
348
|
-
|
|
|
349
|
-
| Selecting a file
|
|
350
|
-
|
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
43
|
+
| The question is about… | Use | Why |
|
|
44
|
+
|------------------------------------------------------------------|--------------------------------------------------------------|---------------------------------------------------------------------------------------------|
|
|
45
|
+
| Element existence / text / role | `dom --filter` | Cheapest, structured, no vision tokens |
|
|
46
|
+
| Count of matching elements | `dom --filter X --count` | Single integer, no tree output, no ref mutations |
|
|
47
|
+
| App state, store contents, computed values | `eval "expr"` | DOM doesn't expose JS state; reading the tree to infer it is wasteful and unreliable |
|
|
48
|
+
| Does `window.X` / a globally-exposed API exist? | `eval "typeof window.X"` | DOM doesn't show JS globals; only authoritative check |
|
|
49
|
+
| An element that has not rendered yet | `wait --filter "<text>"` | Exits on appearance and non-zero on timeout — a real gate, unlike a fixed pause |
|
|
50
|
+
| State *trajectory* — what changed during/after an action | `watch "expr" --until …` or `--max-changes 1` | `eval` shows the final snapshot only; `watch` shows the diffs in order |
|
|
51
|
+
| Worker logic (SharedWorker / ServiceWorker) | `eval --target <name>` | Workers have no DOM at all |
|
|
52
|
+
| Did the last action throw or warn? | `console --clear` before, `console --level error,warn` after | Catches errors that don't surface in the DOM |
|
|
53
|
+
| Did an expected API call fire, and with what status? | `network --clear` before, `network --url "<glob>"` after | Captures eagerly from launch; the DOM shows the result, not the call |
|
|
54
|
+
| What happened over a long / flaky / reload-spanning run | `logs start` … `logs tail --grep`/`--since` | Durable one-line-per-record timeline of page + workers; `console` loses it on idle shutdown |
|
|
55
|
+
| Layout/visual of a specific element | `screenshot --crop "<element>"` | ~1.6k tokens (1 tile) — crops to bounding box, massive token win |
|
|
56
|
+
| Layout, spacing, full-window visual regression | `screenshot --scale 0.5` | The only tool that sees pixels — but expensive (~6k tokens), use last |
|
|
57
|
+
| Canvas/WebGL scene contents | `scene --diff` | DOM is empty for canvas apps |
|
|
58
|
+
| What DOM nodes changed after an interaction | `dom --diff` | Returns only `+`/`-` lines; much cheaper than re-reading the full tree |
|
|
59
|
+
| Selecting a file for an input that exists in the DOM | `upload --selector` | No picker opens at all; works on hidden inputs, which have no ref |
|
|
60
|
+
| Selecting a file when the input appears only mid-click | `dialog arm --file` then `click` | The only way — the input does not exist before the click and is gone after |
|
|
61
|
+
| Does any user action reach this code? | `coverage --clear` before, `coverage --file X` after | The only tool that answers it; a diff cannot |
|
|
62
|
+
| What handler is bound to this element, and where is it declared? | `listeners --filter "<text>"` | Gives `file:line` without reading the source |
|
|
63
|
+
| The window stopped responding after a click | `dialog` | Shows whether a modal was answered, and what the app was told |
|
|
64
|
+
|
|
65
|
+
When two tools could answer the same question, prefer the one higher up the table.
|
|
66
|
+
|
|
67
|
+
## Execution discipline (read first, every run)
|
|
68
|
+
|
|
69
|
+
A run produces one of three outcomes per step: **pass**, **fail**, **requires_visual_review**. There is no fourth bucket
|
|
70
|
+
called "actually fine, here's why". A failed `Expected:` line is FAIL.
|
|
359
71
|
|
|
360
72
|
**How to run the commands themselves:**
|
|
361
73
|
|
|
362
|
-
- **Never discard output.** No `>/dev/null`, no `2>&1` to nowhere, no `| head -1` on a command whose
|
|
363
|
-
|
|
364
|
-
|
|
74
|
+
- **Never discard output.** No `>/dev/null`, no `2>&1` to nowhere, no `| head -1` on a command whose failure you have
|
|
75
|
+
not yet read. Every agent-view command prints either the evidence or the reason it failed; a suppressed `click` that
|
|
76
|
+
matched nothing looks exactly like a successful one.
|
|
365
77
|
- **Chain with `&&`, not newlines.** Newline-separated commands keep running after a failure, so a
|
|
366
78
|
broken first step is followed by three steps acting on the wrong state. `&&` stops at the first
|
|
367
79
|
non-zero exit.
|
|
368
80
|
- **One action, then one check.** `click` is not evidence. The evidence is the `dom --diff`,
|
|
369
|
-
`dom --filter … --count`, `eval`, or `console --level error` you run after it.
|
|
370
|
-
|
|
371
|
-
- **
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
81
|
+
`dom --filter … --count`, `eval`, or `console --level error` you run after it. Three clicks in a row with no check
|
|
82
|
+
between them prove nothing about any of them.
|
|
83
|
+
- **Never sleep for a fixed time.** No `sleep`, no `timeout`, no `ping -n N 127.0.0.1` as a delay (agents reach for
|
|
84
|
+
`ping` when the harness blocks `sleep` — the same mistake in worse clothing). A fixed pause is either too short, and
|
|
85
|
+
you assert against a half-rendered UI, or too long, and you burn wall-clock on every step. There is a condition-based
|
|
86
|
+
wait for every kind of signal:
|
|
87
|
+
|
|
88
|
+
| You are waiting for… | Use |
|
|
89
|
+
|---|---|
|
|
90
|
+
| An element to render | `wait --filter "<text>"` |
|
|
91
|
+
| A store/state value | `watch "<expr>" --until "<expr>"` |
|
|
92
|
+
| A log line | `console --follow --until "<pattern>"` |
|
|
93
|
+
| A request to fire | `network --follow --until "<url>"` |
|
|
94
|
+
|
|
95
|
+
If none of these fits, poll `dom --filter X --count` with an explicit attempt cap and report how many attempts it
|
|
96
|
+
took.
|
|
97
|
+
- **Call the `agent-view` binary.** Install it once in the target project (`pnpm add -D @petukhovart/agent-view`, or
|
|
98
|
+
your package manager's equivalent) and call
|
|
99
|
+
`agent-view …` / `pnpm exec agent-view …`. Prefixing every call with `npx <package>` re-resolves the package on each
|
|
100
|
+
invocation, and in permission-gated harnesses each call then needs a fresh approval prompt.
|
|
101
|
+
|
|
102
|
+
These heuristics catch real bugs. Skipping them is how a run silently passes while the bug sits in plain sight in the
|
|
103
|
+
same data:
|
|
104
|
+
|
|
105
|
+
1. **A failed expectation is FAIL.** If output disagrees with what the step expected, mark `fail` and continue. Do not
|
|
106
|
+
soften the expectation. Do not invent prose explanations inline ("label reuse",
|
|
107
|
+
"convention", "arithmetic off"). Justifications belong in the bug report after the run, never in the per-step log.
|
|
108
|
+
|
|
109
|
+
2. **UI-vs-model mismatch is the bug, not noise.** When a count or hierarchy check returns
|
|
110
|
+
`match: false`, the default hypothesis is that the UI renderer is wrong. Before reaching for "the filter matched
|
|
111
|
+
something extra in a side panel", query the bounding boxes and ancestor chains of the matched elements — two matches
|
|
112
|
+
at the same x in adjacent y rows are sibling rows in one list, i.e. a renderer bug. The model is one representation,
|
|
113
|
+
not the source of truth; the bug may live in the gap between model and UI.
|
|
114
|
+
|
|
115
|
+
3. **Defensive eval reads.** Sentinel-check every `node.field` read (`transform.x`,
|
|
116
|
+
`transform.width`) before using it in arithmetic. A renamed field silently returns `NaN`/`null`, which fail-passes
|
|
117
|
+
downstream comparisons. Add `isFinite(value)` / `value !== undefined` guards inline.
|
|
118
|
+
|
|
119
|
+
4. **No hardcoded literal IDs.** A hardcoded node-ID prefix that no longer matches the current scene degrades the whole
|
|
120
|
+
check to a silent no-op. Verify at least one expected ID exists; if not, derive IDs by role at runtime, proceed with
|
|
121
|
+
the corrected lookup, and say the plan needs an ID refresh.
|
|
122
|
+
|
|
123
|
+
5. **Reload checkpoint is not optional.** If the feature mutated persisted structure, run one:
|
|
124
|
+
`agent-view eval "location.reload()"`, wait for the app to come back, re-read the structural signature, diff. Drift
|
|
125
|
+
is a real bug, not a "fixed-up on save".
|
|
126
|
+
|
|
127
|
+
6. **Invariants run first or fail closed.** When the plan states invariants, execute those steps before the
|
|
128
|
+
action-specific checks. A failed invariant is FAIL for that invariant *and* a flag on the rest of the run — keep
|
|
129
|
+
running the remaining steps, tagged "trust-impaired until invariant restored".
|
|
130
|
+
|
|
131
|
+
7. **Never claim a `window.*` API is missing without `eval`.** Before reporting "API not exposed" /
|
|
132
|
+
"global X doesn't exist" / "the host doesn't expose Y", run `agent-view eval "typeof window.X"`
|
|
133
|
+
and report the literal result (`"undefined"` / `"object"` / `"function"`). DOM scraping cannot answer this — globals
|
|
134
|
+
are not in the AX tree. If it returns `"undefined"` the API really is absent from the main world; anything else means
|
|
135
|
+
the API is reachable and your earlier conclusion was wrong. No exceptions, no "I checked the source code instead".
|
|
379
136
|
|
|
380
|
-
|
|
381
|
-
- Default hypothesis: the UI renderer is wrong.
|
|
382
|
-
- Before considering "the filter matched something extra in some side panel", query bounding boxes and ancestor chains of the matched elements. Two matches at the same x-coordinate in adjacent y rows = sibling rows in one list = renderer bug.
|
|
383
|
-
- Do not dismiss DOM/model divergence with "scene-graph is the source of truth". The model is one representation; the bug may live in the gap between model and UI.
|
|
384
|
-
|
|
385
|
-
3. **Defensive eval reads.** Every `node.field` read (e.g. `transform.x`, `transform.width`) must be sentinel-checked before being used in arithmetic. A renamed field silently returns `NaN`/`null`, which fail-passes downstream comparisons. Add `isFinite(value)` / `value !== undefined` guards inline.
|
|
386
|
-
|
|
387
|
-
4. **No hardcoded literal IDs.** A hardcoded node-ID prefix that no longer matches the current scene degrades the whole check to a silent no-op. Verify at least one expected ID exists; if not, derive IDs by role at runtime and proceed with the corrected lookup, and say the plan needs an ID refresh.
|
|
388
|
-
|
|
389
|
-
5. **Reload checkpoint is not optional.** If the feature mutated persisted structure, run one: `agent-view eval "location.reload()"`, wait for the app to come back, re-read the structural signature, diff. Drift is a real bug, not a "fixed-up on save".
|
|
390
|
-
|
|
391
|
-
6. **Invariants run first or fail closed.** When the plan states invariants, execute those steps before the action-specific checks. A failed invariant is FAIL for that invariant *and* a flag on the rest of the run — keep running the remaining steps, tag them as "trust-impaired until invariant restored".
|
|
137
|
+
## Verification Workflow
|
|
392
138
|
|
|
393
|
-
|
|
139
|
+
Run the whole thing inline — **no subagent**. Resolve the window id once with `agent-view discover`
|
|
140
|
+
if you need `--window`.
|
|
394
141
|
|
|
395
|
-
### Ad-hoc
|
|
142
|
+
### Ad-hoc mode (standalone)
|
|
396
143
|
|
|
397
144
|
After making code changes:
|
|
398
145
|
|
|
399
|
-
1. **Determine affected areas** from git diff
|
|
400
|
-
2. **Ensure app is running**: `agent-view launch` or `agent-view discover`
|
|
401
|
-
3. **Inspect DOM**: `agent-view dom --filter "<area>" --depth 2` —
|
|
402
|
-
4. **Interact
|
|
403
|
-
5. **For canvas apps**: `agent-view scene --diff
|
|
404
|
-
6. **For non-DOM truth** (store, computed values, worker state): `agent-view eval
|
|
405
|
-
7. **
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
This mode works with any workflow that generates plan files with UI scenarios.
|
|
146
|
+
1. **Determine affected areas** from `git diff` — every changed file that renders or drives UI needs at least one check.
|
|
147
|
+
2. **Ensure the app is running**: `agent-view launch` (or `agent-view discover`).
|
|
148
|
+
3. **Inspect DOM**: `agent-view dom --filter "<area>" --depth 2` — structure matches expectations.
|
|
149
|
+
4. **Interact**: `agent-view click`/`fill` → `agent-view dom --filter` to verify the state changed.
|
|
150
|
+
5. **For canvas apps**: `agent-view scene --diff`.
|
|
151
|
+
6. **For non-DOM truth** (store, computed values, worker state): `agent-view eval`.
|
|
152
|
+
7. **Before claiming code is unreachable**: `agent-view coverage --clear` → the action →
|
|
153
|
+
`agent-view coverage --file "<file>"`. Reading the diff is not evidence either way; an empty result narrows the claim
|
|
154
|
+
to "this action does not reach it".
|
|
155
|
+
8. **After any interaction that could fail silently**: `agent-view console --level error` — catches uncaught exceptions,
|
|
156
|
+
network failures, framework warnings.
|
|
157
|
+
9. **Screenshot last, for visual confirm only**: `agent-view screenshot --scale 0.5`.
|
|
158
|
+
|
|
159
|
+
### Scenario mode (from a plan)
|
|
160
|
+
|
|
161
|
+
When UI scenarios are pre-generated (e.g. a plan file with a `## UI Scenarios` section): read the steps, resolve each
|
|
162
|
+
symbolic `$var` via `agent-view dom --filter "<text>" --depth 3` to a ref, execute the steps in order, and verify each
|
|
163
|
+
expected outcome with `dom --filter`. Screenshot only on FAIL and at the end of an E2E scenario, never per step.
|
|
419
164
|
|
|
420
165
|
### Reporting
|
|
421
166
|
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
Report a tight summary: passed / failed / visual-review counts, one line per failure, invariant violations called out separately. Don't paste raw stdout unless asked. After 2–3 consecutive failures, stop and distinguish "the plan is stale" (hardcoded IDs no longer match the current UI) from "the feature is broken" (invariants violated on a current scene) — they need opposite fixes.
|
|
425
|
-
|
|
426
|
-
### Design Conformance (inline)
|
|
167
|
+
One line per step, so the report stays machine-readable:
|
|
427
168
|
|
|
428
|
-
|
|
169
|
+
```
|
|
170
|
+
<step label> | pass | fail | requires_visual_review — <evidence command and its result>
|
|
171
|
+
```
|
|
429
172
|
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
2. `Read` both the captured image and the `expected_path`. If `expected_path` doesn't exist or is unreadable, mark the pair `skipped (expected_missing)` and move on.
|
|
433
|
-
3. Compare visually for: layout (relative position, alignment), sizing, color (dominant color family), typography (weight/size broadly), content presence (anything missing or extra), decorations (borders, shadows, dashed/solid lines, icons).
|
|
434
|
-
4. Report each pair as `match` / `minor_mismatch` / `major_mismatch` with a one-sentence deviation. Major = missing/wrong component, broken layout, wrong color family, wrong text content. Minor = <10px spacing drift, slight color shade, small decoration difference.
|
|
173
|
+
Close with passed / failed / visual-review counts, and call out invariant violations separately. Do not paste raw stdout
|
|
174
|
+
unless asked.
|
|
435
175
|
|
|
436
|
-
|
|
176
|
+
**Design conformance** — when you are handed `(label, screenshot command, expected reference path)`
|
|
177
|
+
rows, follow [`references/design-conformance.md`](references/design-conformance.md).
|
|
437
178
|
|
|
438
179
|
## Resilience
|
|
439
180
|
|
|
440
|
-
- **
|
|
441
|
-
|
|
442
|
-
- **
|
|
443
|
-
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
- **
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
- **Lazy server**: auto-starts on first call, shuts down after 5min idle
|
|
181
|
+
- **Element not found:** `agent-view wait --filter "<text>" --timeout 5` covers the render delay after HMR. If it times
|
|
182
|
+
out — report FAIL.
|
|
183
|
+
- **Stale refs:** re-run `dom` after HMR, navigation, or a state change before interacting again.
|
|
184
|
+
- **CDP disconnect:** `agent-view discover` to check. If no windows — `agent-view launch`. On
|
|
185
|
+
`PORT_CONFLICT` the CLI reports the owning PID and process name; surface it and ask the user to free the port. Never
|
|
186
|
+
kill a foreign process from this skill.
|
|
187
|
+
- **`CDP_TIMEOUT`:** the command hit the server-side deadline and cached sessions for that port were dropped, so retry
|
|
188
|
+
once. Repeated timeouts mean the app's DevTools endpoint is wedged — restart it.
|
|
189
|
+
- **Retry budget: 2 per command**, then SKIP the step with a warning. Two covers a transient CDP drop; a third repeat
|
|
190
|
+
means the app or the plan is wrong, not the call. After two or three consecutive failures, stop and distinguish "the
|
|
191
|
+
plan is stale" (hardcoded IDs no longer match the UI) from "the feature is broken" (invariants violated on a current
|
|
192
|
+
scene) — they need opposite fixes.
|
|
453
193
|
|
|
454
194
|
## Token Optimization
|
|
455
195
|
|
|
456
|
-
Vision tokens dominate cost
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|---|---|
|
|
460
|
-
| `agent-view eval "expr"` for state checks | Returns one value (~50 tokens) instead of a DOM/screenshot |
|
|
461
|
-
| `agent-view dom --filter "row" --count` | Single integer answer — zero tree tokens |
|
|
462
|
-
| `agent-view dom --filter X --depth 2` | Narrow tree to relevant subtree, cap depth |
|
|
463
|
-
| `agent-view screenshot --scale 0.5` | ~3× fewer vision tokens (4 tiles) |
|
|
464
|
-
| `agent-view screenshot --scale 0.25` | ~12× fewer vision tokens (1 tile, ~1.6k tokens) |
|
|
465
|
-
| `agent-view screenshot --crop "<element>"` | ~12× fewer in best case (1 tile) — crops to element bounding box |
|
|
466
|
-
| DOM/eval-first: screenshot only for final visual confirm | Eliminates most screenshot calls |
|
|
196
|
+
Vision tokens dominate cost: a full-res screenshot is ≈19k tokens (1920×1080, 12 tiles), `--scale 0.5`
|
|
197
|
+
≈6k (4 tiles), `--scale 0.25` and `--crop` ≈1.6k (1 tile). A text answer is ~50. So `--depth` and
|
|
198
|
+
`--filter` on `dom`, and `--count` where a number is the whole answer, cost near nothing by comparison.
|
|
467
199
|
|
|
468
|
-
**Default rule**:
|
|
200
|
+
**Default rule**: the answer is a value → `eval`; the answer is "is element X visible/correct?" →
|
|
201
|
+
`dom --filter`; you need pixels for one section → `screenshot --crop "<element>"`; only call
|
|
202
|
+
`screenshot --scale 0.5` for full-window visual proof.
|