@petukhovart/agent-view 0.3.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +14 -0
- package/.claude-plugin/plugin.json +2 -2
- package/README.md +292 -143
- package/agents/design-conformance-runner.md +101 -0
- package/agents/verify-runner.md +97 -0
- package/dist/cdp/ax-cache.d.ts +7 -0
- package/dist/cdp/ax-cache.d.ts.map +1 -1
- package/dist/cdp/ax-cache.js +7 -0
- package/dist/cdp/ax-cache.js.map +1 -1
- package/dist/cdp/transport.d.ts.map +1 -1
- package/dist/cdp/transport.js +98 -27
- package/dist/cdp/transport.js.map +1 -1
- package/dist/cdp/types.d.ts +45 -2
- package/dist/cdp/types.d.ts.map +1 -1
- package/dist/cdp/types.js +6 -0
- package/dist/cdp/types.js.map +1 -1
- package/dist/cli/commands/console.d.ts +1 -0
- package/dist/cli/commands/console.d.ts.map +1 -1
- package/dist/cli/commands/console.js +1 -0
- package/dist/cli/commands/console.js.map +1 -1
- package/dist/cli/commands/dom.d.ts +4 -0
- package/dist/cli/commands/dom.d.ts.map +1 -1
- package/dist/cli/commands/dom.js +4 -0
- package/dist/cli/commands/dom.js.map +1 -1
- package/dist/cli/commands/drag.d.ts +13 -0
- package/dist/cli/commands/drag.d.ts.map +1 -0
- package/dist/cli/commands/drag.js +72 -0
- package/dist/cli/commands/drag.js.map +1 -0
- package/dist/cli/commands/scene.d.ts +1 -0
- package/dist/cli/commands/scene.d.ts.map +1 -1
- package/dist/cli/commands/scene.js +2 -0
- package/dist/cli/commands/scene.js.map +1 -1
- package/dist/cli/commands/screenshot.d.ts +1 -0
- package/dist/cli/commands/screenshot.d.ts.map +1 -1
- package/dist/cli/commands/screenshot.js +4 -0
- package/dist/cli/commands/screenshot.js.map +1 -1
- package/dist/cli/commands/snap.d.ts +1 -0
- package/dist/cli/commands/snap.d.ts.map +1 -1
- package/dist/cli/commands/snap.js +2 -0
- package/dist/cli/commands/snap.js.map +1 -1
- package/dist/cli/commands/watch.d.ts +12 -0
- package/dist/cli/commands/watch.d.ts.map +1 -0
- package/dist/cli/commands/watch.js +233 -0
- package/dist/cli/commands/watch.js.map +1 -0
- package/dist/cli/index.js +48 -1
- package/dist/cli/index.js.map +1 -1
- package/dist/config/manager.js +1 -1
- package/dist/config/types.d.ts +0 -3
- package/dist/config/types.d.ts.map +1 -1
- package/dist/inspectors/dom/index.d.ts +13 -0
- package/dist/inspectors/dom/index.d.ts.map +1 -1
- package/dist/inspectors/dom/index.js +255 -103
- package/dist/inspectors/dom/index.js.map +1 -1
- package/dist/inspectors/scene/formatter.d.ts.map +1 -1
- package/dist/inspectors/scene/formatter.js +56 -8
- package/dist/inspectors/scene/formatter.js.map +1 -1
- package/dist/inspectors/scene/types.d.ts +1 -0
- package/dist/inspectors/scene/types.d.ts.map +1 -1
- package/dist/inspectors/watch/index.d.ts +18 -0
- package/dist/inspectors/watch/index.d.ts.map +1 -0
- package/dist/inspectors/watch/index.js +39 -0
- package/dist/inspectors/watch/index.js.map +1 -0
- package/dist/inspectors/watch/types.d.ts +70 -0
- package/dist/inspectors/watch/types.d.ts.map +1 -0
- package/dist/inspectors/watch/types.js +21 -0
- package/dist/inspectors/watch/types.js.map +1 -0
- package/dist/server/server.d.ts +9 -1
- package/dist/server/server.d.ts.map +1 -1
- package/dist/server/server.js +279 -48
- package/dist/server/server.js.map +1 -1
- package/dist/server/watch-session.d.ts +32 -0
- package/dist/server/watch-session.d.ts.map +1 -0
- package/dist/server/watch-session.js +165 -0
- package/dist/server/watch-session.js.map +1 -0
- package/dist/types.d.ts +2 -0
- package/dist/types.d.ts.map +1 -1
- package/package.json +62 -60
- package/skills/verify/SKILL.md +84 -13
- package/skills/verify-recipe/SKILL.md +202 -0
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "agent-view",
|
|
3
|
+
"owner": {
|
|
4
|
+
"name": "PetukhovArt",
|
|
5
|
+
"url": "https://github.com/PetukhovArt"
|
|
6
|
+
},
|
|
7
|
+
"plugins": [
|
|
8
|
+
{
|
|
9
|
+
"name": "agent-view",
|
|
10
|
+
"source": "./",
|
|
11
|
+
"description": "Visual verification CLI for desktop apps (Electron/Tauri/Browser) via Chrome DevTools Protocol. Inspect DOM, interact with UI, capture screenshots, and read WebGL scene graphs."
|
|
12
|
+
}
|
|
13
|
+
]
|
|
14
|
+
}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-view",
|
|
3
|
-
"description": "Visual verification CLI for desktop apps (Electron/Tauri/Browser) via Chrome DevTools Protocol.
|
|
4
|
-
"version": "0.
|
|
3
|
+
"description": "Visual verification CLI for desktop apps (Electron/Tauri/Browser) via Chrome DevTools Protocol. Ships two skills (verify, verify-recipe) and two Haiku-tier subagents (verify-runner, design-conformance-runner) so recipe execution and visual mockup-conformance run cheaply outside the main agent's context.",
|
|
4
|
+
"version": "0.6.0",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"cdp",
|
|
7
7
|
"electron",
|
package/README.md
CHANGED
|
@@ -1,13 +1,37 @@
|
|
|
1
1
|
# agent-view
|
|
2
2
|
|
|
3
|
-
**Give your AI agent eyes and hands for desktop apps
|
|
3
|
+
**Give your AI agent eyes and hands for complex desktop apps verification**
|
|
4
4
|
|
|
5
5
|
AI coding agents can write code, run tests, and read logs — but they can't *see* what the app actually looks like. Without visual verification, an agent is essentially **coding blind** — builds pass, tests are green, but the login form is broken, the button is off-screen, or the modal never appears.
|
|
6
6
|
|
|
7
|
-
agent-view bridges that gap: it connects to any Chromium-based desktop app via Chrome DevTools Protocol and lets the agent inspect
|
|
7
|
+
agent-view bridges that gap: it connects to any Chromium-based desktop app via Chrome DevTools Protocol and lets the agent inspect, interact, and verify.
|
|
8
8
|
|
|
9
9
|
Built for [Claude Code](https://docs.anthropic.com/en/docs/claude-code), but works with any AI agent or automation pipeline that can call CLI commands.
|
|
10
10
|
|
|
11
|
+
## 5-minute quickstart
|
|
12
|
+
|
|
13
|
+
1. **[Install](#install--update)** — `npm i -g @petukhovart/agent-view` plus the Claude Code plugin.
|
|
14
|
+
2. **[Enable CDP](#enabling-cdp) in your app** — one line in your Electron `main.ts`, gated to dev builds.
|
|
15
|
+
3. **`agent-view init`** in your project — then add `"allowEval": true` to the generated `agent-view.config.json`.
|
|
16
|
+
4. **`agent-view launch`** — starts your app and waits for CDP readiness.
|
|
17
|
+
5. **In Claude Code:** describe what you want verified — see [example prompts](#recommended-workflow-with-claude-code).
|
|
18
|
+
|
|
19
|
+
Each step links to the full explanation below.
|
|
20
|
+
|
|
21
|
+
## What it does
|
|
22
|
+
|
|
23
|
+
- **DOM accessibility tree** with ref IDs — compact, LLM-friendly, with text/role filters; `--compact` merges single-child chains for 40–60% fewer tokens, `--count` returns just the match count, `--max-lines` caps output, `--diff` emits only what changed since the last call
|
|
24
|
+
- **Screenshots** — full-res PNG, scaled WebP (~3–12× fewer vision tokens), or `--crop <filter>` to a single element bounding box
|
|
25
|
+
- **Interaction** — click, fill, and drag by ref or coordinates; works with Vue/React/native frameworks
|
|
26
|
+
- **JS state via `eval`** — read store contents, computed values, async results without scraping the DOM
|
|
27
|
+
- **Reactive state via `watch`** — stream JSON-patch diffs of any expression (store, ref, computed) until a condition is met
|
|
28
|
+
- **Console capture** — `console.log/warn/error` per page and per worker, with level/since filters and `--follow --until <pattern>` for early exit on a matching log
|
|
29
|
+
- **Worker access** — SharedWorker, ServiceWorker, dedicated Worker visible alongside pages; fuzzy `--target` resolution everywhere (id → title → URL)
|
|
30
|
+
- **Canvas / WebGL scene graph** — PixiJS today, engine-pluggable; `--compact` mirrors the DOM mode
|
|
31
|
+
- **Design-conformance verification** — pair screenshot commands with local design references (Figma export, hand-off
|
|
32
|
+
PNGs, any image on disk) inside a verify-recipe; the `design-conformance-runner` subagent reports visual deviations
|
|
33
|
+
against the mockup
|
|
34
|
+
|
|
11
35
|
## Why CLI, not MCP?
|
|
12
36
|
|
|
13
37
|
Most alternatives in this space are MCP servers with 30+ tool definitions loaded into context on every session. That burns tokens before the agent even starts working.
|
|
@@ -16,68 +40,75 @@ agent-view is a CLI. One Bash call, compact text output, zero schema overhead. T
|
|
|
16
40
|
|
|
17
41
|
And CLI works everywhere — Claude Code, Copilot, Codex, custom pipelines, CI. No MCP client required.
|
|
18
42
|
|
|
19
|
-
##
|
|
43
|
+
## Install & Update
|
|
20
44
|
|
|
21
|
-
|
|
45
|
+
### Claude Code (recommended)
|
|
22
46
|
|
|
23
|
-
|
|
24
|
-
Code → Launch → See → Verify → Fix → See again
|
|
25
|
-
```
|
|
47
|
+
Two steps — both required:
|
|
26
48
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
Beyond DOM and screenshots, v0.3.0 adds direct access to JS state and console — including SharedWorker / ServiceWorker. Useful when «what's rendered» isn't enough and you need to check what's actually in the store, in a worker, or in the console.
|
|
49
|
+
```bash
|
|
50
|
+
# 1. Install the plugin — adds two skills: verify (run checks against a live app) and verify-recipe (author a verification plan)
|
|
51
|
+
/plugin marketplace add PetukhovArt/agent-view
|
|
52
|
+
/plugin install agent-view@agent-view
|
|
32
53
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
- `targets` — list every connectable CDP target.
|
|
54
|
+
# 2. Install the CLI — the skill calls these binaries; the plugin doesn't bundle them
|
|
55
|
+
npm install -g @petukhovart/agent-view
|
|
56
|
+
```
|
|
37
57
|
|
|
38
|
-
|
|
58
|
+
The plugin ships two skills. **`verify`** executes visual and runtime checks against a running app. **`verify-recipe`** generates a `.claude/verify-recipes/<slug>.md` file — a disciplined, cheapest-first command sequence for a feature or bugfix — that you or any AI agent can run later. Trigger it with phrases like "write a verify-recipe for the login fix" or "generate a verification plan for this feature".
|
|
39
59
|
|
|
40
|
-
|
|
60
|
+
For Haiku-cost recipe execution and visual conformance against design mockups,
|
|
61
|
+
see [Recommended workflow with Claude Code](#recommended-workflow-with-claude-code) below.
|
|
41
62
|
|
|
42
|
-
|
|
63
|
+
Verify:
|
|
43
64
|
|
|
44
|
-
|
|
65
|
+
```bash
|
|
66
|
+
agent-view --version # 0.5.0+
|
|
67
|
+
```
|
|
45
68
|
|
|
46
|
-
|
|
47
|
-
|---|---|---|---|
|
|
48
|
-
| `dom` cold fetch | 10ms | 2ms | ~30–80ms |
|
|
49
|
-
| `dom` warm (cache hit) | 10ms | 1ms | ~30–80ms |
|
|
50
|
-
| Full cycle `dom → click → dom` | 27ms | 17ms | ~75ms |
|
|
51
|
-
| `click --filter "Save"` | 12ms | 17ms† | ~15–30ms |
|
|
69
|
+
### Standalone CLI (any other agent, CI, or scripting)
|
|
52
70
|
|
|
53
|
-
|
|
54
|
-
|
|
71
|
+
```bash
|
|
72
|
+
npm install -g @petukhovart/agent-view
|
|
73
|
+
```
|
|
55
74
|
|
|
56
|
-
|
|
75
|
+
Everything works from this alone — `agent-view dom`, `screenshot`, `eval`, etc. Skip the plugin step.
|
|
57
76
|
|
|
58
|
-
|
|
77
|
+
### Update
|
|
59
78
|
|
|
60
|
-
|
|
79
|
+
```bash
|
|
80
|
+
# Update the CLI to the latest version
|
|
81
|
+
npm update -g @petukhovart/agent-view
|
|
61
82
|
|
|
62
|
-
|
|
83
|
+
# Update the Claude Code plugin (refreshes skills from the marketplace)
|
|
84
|
+
/plugin marketplace update agent-view
|
|
85
|
+
/plugin install agent-view@agent-view # re-run to pick up new skill versions
|
|
86
|
+
```
|
|
63
87
|
|
|
64
|
-
|
|
88
|
+
The two are independent — bump the CLI when a new release ships features (see `CHANGELOG.md`), bump the plugin when skill instructions change.
|
|
65
89
|
|
|
66
90
|
## Enabling CDP
|
|
67
91
|
|
|
92
|
+
agent-view talks to your app over Chrome DevTools Protocol. Your app must be launched with a debugging port open.
|
|
93
|
+
|
|
68
94
|
### Recommended: in code (reliable, works with any build tool)
|
|
69
95
|
|
|
70
|
-
Add to your Electron main process:
|
|
96
|
+
Add to your Electron main process, **before `app.whenReady()`** (top of `main.ts`/`main.js`, right after the `electron` import — switches set after the app is ready are ignored):
|
|
71
97
|
|
|
72
98
|
```js
|
|
73
|
-
app
|
|
99
|
+
import { app } from 'electron';
|
|
100
|
+
|
|
101
|
+
app.commandLine.appendSwitch('remote-debugging-port', '9876');
|
|
74
102
|
```
|
|
75
103
|
|
|
76
|
-
|
|
104
|
+
> Any free port works — `9876` is just an example. Avoid `9222` (Chrome's own default remote-debugging port) to prevent
|
|
105
|
+
> collisions when Chrome is open.
|
|
106
|
+
|
|
107
|
+
**Production safety:** an open CDP port in a signed/notarized build is a remote-code-execution surface. Gate it on `!app.isPackaged` so it only opens in dev:
|
|
77
108
|
|
|
78
109
|
```js
|
|
79
110
|
if (!app.isPackaged) {
|
|
80
|
-
|
|
111
|
+
app.commandLine.appendSwitch('remote-debugging-port', '9876');
|
|
81
112
|
}
|
|
82
113
|
```
|
|
83
114
|
|
|
@@ -85,72 +116,163 @@ if (!app.isPackaged) {
|
|
|
85
116
|
|
|
86
117
|
```bash
|
|
87
118
|
# Plain Electron
|
|
88
|
-
electron . --remote-debugging-port=
|
|
119
|
+
electron . --remote-debugging-port=9876
|
|
89
120
|
|
|
90
121
|
# electron-vite (note the -- to forward the flag past the build tool)
|
|
91
|
-
npx electron-vite dev -- --remote-debugging-port=
|
|
122
|
+
npx electron-vite dev -- --remote-debugging-port=9876
|
|
92
123
|
```
|
|
93
124
|
|
|
94
125
|
### Other runtimes
|
|
95
126
|
|
|
96
|
-
| Runtime
|
|
97
|
-
|
|
98
|
-
| **Tauri**
|
|
99
|
-
| **Any Chromium app** | `--remote-debugging-port=
|
|
127
|
+
| Runtime | Setup |
|
|
128
|
+
|----------------------|--------------------------------------------|
|
|
129
|
+
| **Tauri** | CDP via devtools configuration |
|
|
130
|
+
| **Any Chromium app** | `--remote-debugging-port=9876` launch flag |
|
|
100
131
|
|
|
101
132
|
### Verify CDP is working
|
|
102
133
|
|
|
103
134
|
```bash
|
|
104
|
-
curl -s http://localhost:
|
|
135
|
+
curl -s http://localhost:9876/json/version
|
|
105
136
|
```
|
|
106
137
|
|
|
107
|
-
If you see a JSON response with
|
|
138
|
+
If you see a JSON response with process info — you're good.
|
|
108
139
|
|
|
109
|
-
##
|
|
140
|
+
## Quick start with Claude Code (Prompting)
|
|
110
141
|
|
|
111
|
-
|
|
142
|
+
Assumes the plugin and CLI are installed (see [Install & Update](#install--update)) and CDP is enabled in your app (
|
|
143
|
+
see [Enabling CDP](#enabling-cdp)).
|
|
144
|
+
|
|
145
|
+
**1. Generate config once:**
|
|
112
146
|
|
|
113
147
|
```bash
|
|
114
|
-
|
|
148
|
+
cd your-electron-project
|
|
149
|
+
agent-view init # writes agent-view.config.json — auto-detects runtime, port, launch script, webgl engine
|
|
115
150
|
```
|
|
116
151
|
|
|
117
|
-
|
|
152
|
+
Then open `agent-view.config.json` and add `"allowEval": true` if you want recipes to use `eval` / `watch` (most do —
|
|
153
|
+
store/state assertions are 100× cheaper than DOM scraping). Off by default for security; see [Config](#config) for the
|
|
154
|
+
full field list.
|
|
155
|
+
|
|
156
|
+
**2. Start your app:**
|
|
118
157
|
|
|
119
158
|
```bash
|
|
120
|
-
#
|
|
121
|
-
|
|
159
|
+
agent-view launch # uses the `launch` command from config, waits for CDP readiness, idempotent
|
|
160
|
+
```
|
|
122
161
|
|
|
123
|
-
|
|
124
|
-
|
|
162
|
+
(Or start it yourself with `npm run dev` etc. — `launch` is just a convenience.)
|
|
163
|
+
|
|
164
|
+
**3. From Claude Code, ask for a verification:**
|
|
165
|
+
|
|
166
|
+
```text
|
|
167
|
+
The "Save" button on the Settings dialog stayed enabled while a save was in flight.
|
|
168
|
+
I added a `saving` ref bound to :disabled. Verify it works:
|
|
169
|
+
- after click, button must be disabled until network completes
|
|
170
|
+
- no console errors in the flow
|
|
125
171
|
```
|
|
126
172
|
|
|
127
|
-
|
|
173
|
+
Claude picks the `verify` skill, runs a handful of `eval` / `dom --filter` / `console` calls against your live app,
|
|
174
|
+
reports what passed and what failed. No CLI commands typed by you.
|
|
175
|
+
|
|
176
|
+
For a repeatable, multi-step verification (PRD/plan-driven, with optional design-mockup conformance),
|
|
177
|
+
see [Recommended workflow with Claude Code](#recommended-workflow-with-claude-code) — it's the canonical 3-phase prompt
|
|
178
|
+
flow this package is built around.
|
|
128
179
|
|
|
129
|
-
|
|
180
|
+
### Without Claude Code (manual CLI / CI / other agents)
|
|
181
|
+
|
|
182
|
+
If you're scripting from CI or another agent, the CLI works standalone:
|
|
130
183
|
|
|
131
184
|
```bash
|
|
132
|
-
|
|
185
|
+
agent-view discover # List all windows (JSON)
|
|
186
|
+
agent-view dom --filter "Submit" # Accessibility tree, filtered, with ref IDs
|
|
187
|
+
agent-view fill 3 "hello@example.com"
|
|
188
|
+
agent-view click 7
|
|
189
|
+
agent-view eval "store.state.user.role"
|
|
190
|
+
agent-view screenshot --crop "Sidebar" --scale 0.5
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Full surface in [Commands](#commands) below.
|
|
194
|
+
|
|
195
|
+
## Recommended workflow with Claude Code
|
|
196
|
+
|
|
197
|
+
A repeatable, token-efficient flow for "I shipped a feature/fix → confirm it actually works visually and at runtime".
|
|
198
|
+
Three phases, each driven by a focused prompt.
|
|
199
|
+
|
|
200
|
+
### Phase 1 — Author the verification plan (Opus / Sonnet)
|
|
133
201
|
|
|
134
|
-
|
|
135
|
-
|
|
202
|
+
Generate the recipe **once**, from a PRD / plan file / Jira ticket / commit range. The recipe is reusable — re-run after
|
|
203
|
+
every iteration on the same feature.
|
|
136
204
|
|
|
137
|
-
|
|
138
|
-
|
|
205
|
+
```text
|
|
206
|
+
Generate a verify-recipe for the changes in commits <hash1>..<hash2>.
|
|
207
|
+
Source plan: .claude/plans/2026-04-27-login-redirect.md
|
|
208
|
+
Original symptom: after login, redirect went to /home instead of /dashboard.
|
|
139
209
|
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
210
|
+
Design references (if any):
|
|
211
|
+
- /abs/path/figma-exports/login-success.png → label "post-login dashboard"
|
|
212
|
+
- /abs/path/figma-exports/error-state.png → label "invalid creds error"
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
What this triggers: the `verify-recipe` skill interviews you (if more context needed), then writes a
|
|
216
|
+
`.claude/verify-recipes/<slug>.md` with `Repro Steps`, `Evidence Commands` (cheapest-first: `eval` / `dom --filter`
|
|
217
|
+
before `screenshot`), `Regression Checks`, and — if you provided design refs — a `Design Conformance` table mapping
|
|
218
|
+
screenshot commands to expected reference images.
|
|
219
|
+
|
|
220
|
+
**Tip:** if you implemented from a Figma file via the `figma-implement-design` skill, it likely already saved exports
|
|
221
|
+
somewhere on disk — pass those paths. agent-view does NOT fetch from Figma URLs; provide local files only.
|
|
144
222
|
|
|
145
|
-
|
|
146
|
-
agent-view click 5 # Click element by ref
|
|
147
|
-
agent-view fill 3 "hello" # Type into input
|
|
223
|
+
### Phase 2 — Run the recipe (delegated to Haiku)
|
|
148
224
|
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
agent-view screenshot # Visual confirmation
|
|
225
|
+
```text
|
|
226
|
+
Run the verify-recipe at .claude/verify-recipes/<slug>.md.
|
|
152
227
|
```
|
|
153
228
|
|
|
229
|
+
What this triggers: the `verify` skill resolves the window via `agent-view discover`, then spawns the `verify-runner`
|
|
230
|
+
subagent (Haiku) to execute every step and return a compact JSON report. If the recipe has a `Design Conformance`
|
|
231
|
+
section, `design-conformance-runner` (also Haiku) runs in parallel and visually compares screenshots against the
|
|
232
|
+
references.
|
|
233
|
+
|
|
234
|
+
**Effect on cost:** raw output (DOM dumps, screenshots, eval values) stays in the subagent. The main agent only sees the
|
|
235
|
+
merged JSON report — typically <2k tokens vs ~50k if Opus ran the recipe directly.
|
|
236
|
+
|
|
237
|
+
### Phase 3 — Iterate on failures (Opus / Sonnet)
|
|
238
|
+
|
|
239
|
+
The merged report lists `failed`, `requires_visual_review`, and `major_mismatch` items with one-sentence diagnoses. Hand
|
|
240
|
+
any subset back to the main agent:
|
|
241
|
+
|
|
242
|
+
```text
|
|
243
|
+
Three steps failed in the verify run. Fix step 4 (zone filter not mutating store)
|
|
244
|
+
and step 7 (selective filter heading missing). Re-run only those two steps after the fix.
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
The main agent re-runs only the failing commands itself for richer evidence, makes the fix, and re-spawns
|
|
248
|
+
`verify-runner` for the targeted re-check — not the whole recipe.
|
|
249
|
+
|
|
250
|
+
### One-shot prompt (when there's no plan to convert)
|
|
251
|
+
|
|
252
|
+
For small fixes where you don't want a persistent recipe file:
|
|
253
|
+
|
|
254
|
+
```text
|
|
255
|
+
The "Save" button on the Settings dialog wasn't disabling while a save was in flight.
|
|
256
|
+
I added a `saving` ref and bound it to :disabled. Verify it works:
|
|
257
|
+
- after click, button must be disabled until network completes
|
|
258
|
+
- no console errors
|
|
259
|
+
- visual: button greys out (compare to /abs/path/saving-state.png if one is provided)
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
Claude will pick the right skill (usually `verify` ad-hoc mode), run a handful of `eval` / `dom --filter` / `console`
|
|
263
|
+
calls, and only screenshot if the visual claim needs it.
|
|
264
|
+
|
|
265
|
+
### Anti-patterns to avoid
|
|
266
|
+
|
|
267
|
+
- "Just verify the feature" with no plan or symptom — the recipe author can't pick the cheapest signal without knowing
|
|
268
|
+
what "works" means. Give it the symptom that motivated the fix.
|
|
269
|
+
- Pasting Figma URLs and expecting agent-view to download them — it won't. Export the frames you care about to PNG
|
|
270
|
+
first.
|
|
271
|
+
- Running the recipe directly with Opus when a recipe file exists — that's exactly the case `verify-runner` is for.
|
|
272
|
+
Prefer Phase 2's prompt.
|
|
273
|
+
- Stuffing 50 assertions into one recipe — split per-feature. A recipe should run in <2 minutes and produce a report you
|
|
274
|
+
can read in 30 seconds.
|
|
275
|
+
|
|
154
276
|
## How it works
|
|
155
277
|
|
|
156
278
|
```
|
|
@@ -161,28 +283,47 @@ A background server connects to your app's CDP port, caches sessions, and auto-s
|
|
|
161
283
|
|
|
162
284
|
## Config
|
|
163
285
|
|
|
164
|
-
Running `agent-view init` in your project root generates `agent-view.config.json
|
|
286
|
+
Running `agent-view init` in your project root generates `agent-view.config.json`. Minimal form:
|
|
165
287
|
|
|
166
288
|
```json
|
|
167
289
|
{
|
|
168
290
|
"runtime": "electron",
|
|
169
|
-
"port":
|
|
291
|
+
"port": 9876,
|
|
170
292
|
"launch": "npm run dev"
|
|
171
293
|
}
|
|
172
294
|
```
|
|
173
295
|
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
296
|
+
Full form with all optional fields:
|
|
297
|
+
|
|
298
|
+
```json
|
|
299
|
+
{
|
|
300
|
+
"runtime": "electron",
|
|
301
|
+
"port": 9876,
|
|
302
|
+
"launch": "npm run dev",
|
|
303
|
+
"allowEval": true,
|
|
304
|
+
"webgl": {
|
|
305
|
+
"engine": "pixi"
|
|
306
|
+
},
|
|
307
|
+
"consoleBufferSize": 500,
|
|
308
|
+
"consoleTargets": ["page", "shared_worker", "service_worker"]
|
|
309
|
+
}
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
| Field | Required | Description |
|
|
313
|
+
|---------------------|----------|-----------------------------------------------------------------------------------------------------------|
|
|
314
|
+
| `runtime` | yes | `"electron"`, `"tauri"`, or `"browser"` |
|
|
315
|
+
| `port` | yes | CDP debugging port |
|
|
316
|
+
| `launch` | no | Command to start the app |
|
|
317
|
+
| `webgl.engine` | no | `"pixi"` (scene extractor architecture supports adding more engines) |
|
|
318
|
+
| `allowEval` | no | `true` to enable `agent-view eval` and `watch`. Off by default — opt-in for arbitrary JS execution |
|
|
319
|
+
| `consoleBufferSize` | no | Per-target console ring capacity. Default `500` |
|
|
320
|
+
| `consoleTargets` | no | Target types `agent-view console` auto-attaches to. Default `["page", "shared_worker", "service_worker"]` |
|
|
183
321
|
|
|
184
322
|
## Commands
|
|
185
323
|
|
|
324
|
+
Every command targeting a window accepts `--window <id|title-substring>` (IDs come from `discover`). Examples below omit
|
|
325
|
+
it for brevity.
|
|
326
|
+
|
|
186
327
|
### `init`
|
|
187
328
|
|
|
188
329
|
Auto-generates config by reading `package.json`.
|
|
@@ -203,12 +344,22 @@ Dumps the accessibility tree in compact text format. Each element gets a session
|
|
|
203
344
|
agent-view dom
|
|
204
345
|
agent-view dom --filter "Submit" # Filter by text/role
|
|
205
346
|
agent-view dom --depth 3 # Limit tree depth
|
|
206
|
-
agent-view dom --
|
|
347
|
+
agent-view dom --max-lines 200 # Hard line budget (refs for hidden nodes still stored)
|
|
207
348
|
agent-view dom --text # Fall back to DOM textContent search when AX returns no match
|
|
349
|
+
agent-view dom --compact # Merge single-child chains onto one line (saves ~40-60% tokens)
|
|
350
|
+
agent-view dom --count # Return only the count of matching nodes (e.g. "5")
|
|
351
|
+
agent-view dom --filter "row" --count # Count how many rows match
|
|
352
|
+
agent-view dom --diff # Show only lines that changed since last call
|
|
208
353
|
```
|
|
209
354
|
|
|
210
355
|
When `--filter` is set, depth defaults to unlimited so deep matches aren't truncated.
|
|
211
356
|
|
|
357
|
+
`--count` skips tree formatting and ref-store mutations entirely — useful for assertions like "does this section have N rows?" without the token cost of a full tree dump.
|
|
358
|
+
|
|
359
|
+
`--max-lines <n>` caps the number of output lines. When the tree exceeds the budget, output is truncated after `n-1` lines and a summary tail `… M more nodes` is appended. Refs for all nodes — including those past the cutoff — are still registered in the ref store, so a follow-up `dom --filter` or `click <ref>` works without re-running.
|
|
360
|
+
|
|
361
|
+
`--diff` computes a line-level diff against the previous `dom` call for the same target. The first call always returns the full tree (no prior snapshot). Subsequent calls emit only added (`+ `) and removed (`- `) lines. Returns `No changes` when the tree is identical.
|
|
362
|
+
|
|
212
363
|
### `click`
|
|
213
364
|
|
|
214
365
|
Clicks a DOM element by ref ID or coordinates.
|
|
@@ -226,18 +377,40 @@ Types text into an input. Uses native value setter + dispatches input/change eve
|
|
|
226
377
|
agent-view fill 3 "hello@example.com"
|
|
227
378
|
```
|
|
228
379
|
|
|
380
|
+
### `drag`
|
|
381
|
+
|
|
382
|
+
HTML5 / pointer-driven drag-and-drop via CDP `Input.dispatchMouseEvent`
|
|
383
|
+
(`mousePressed` → N × `mouseMoved` → `mouseReleased`). Real mouse events, not
|
|
384
|
+
synthesized JS events — works with `vue-draggable-resizable`, `react-grid-layout`,
|
|
385
|
+
gridstack, kanban boards, file drop zones, map pin drags, resize handles.
|
|
386
|
+
|
|
387
|
+
```bash
|
|
388
|
+
agent-view drag --from 42 --to 88 # ref → ref
|
|
389
|
+
agent-view drag --from-pos 86,792 --to-pos 640,200 # coord → coord (canvas, custom DnD)
|
|
390
|
+
agent-view drag --from 42 --to-pos 640,200 # mixed
|
|
391
|
+
agent-view drag --from 5 --to 9 --steps 20 --hold-ms 150
|
|
392
|
+
```
|
|
393
|
+
|
|
394
|
+
`--steps` (default 10) controls intermediate `mouseMoved` events so libraries
|
|
395
|
+
that throttle on movement deltas still see continuous motion. `--hold-ms`
|
|
396
|
+
inserts a pause between press and the first move (some libs require >100ms
|
|
397
|
+
for touch-style activation). `--button` accepts `left|right|middle`.
|
|
398
|
+
|
|
229
399
|
### `screenshot`
|
|
230
400
|
|
|
231
|
-
Captures a screenshot, saves to temp dir, prints the file path. PNG by default;
|
|
401
|
+
Captures a screenshot, saves to temp dir, prints the file path. PNG by default; WebP (q=80) when `--scale` is set (JPEG fallback for older Chrome/Electron).
|
|
232
402
|
|
|
233
403
|
```bash
|
|
234
404
|
agent-view screenshot
|
|
235
|
-
agent-view screenshot --
|
|
236
|
-
agent-view screenshot --scale 0.
|
|
237
|
-
agent-view screenshot --
|
|
405
|
+
agent-view screenshot --scale 0.5 # Half-res WebP (~3× fewer vision tokens)
|
|
406
|
+
agent-view screenshot --scale 0.25 # Quarter-res WebP (~12× fewer, 1 tile)
|
|
407
|
+
agent-view screenshot --crop "Sidebar" # Crop to element bounding box (~12× fewer in best case)
|
|
408
|
+
agent-view screenshot --crop "Chart" --scale 0.5 # Crop + scale (stacks)
|
|
238
409
|
```
|
|
239
410
|
|
|
240
|
-
`--scale` accepts a factor in `(0, 1]`. CDP-side clip +
|
|
411
|
+
`--scale` accepts a factor in `(0, 1]`. CDP-side clip + WebP encode — recommended for agent loops where vision tokens dominate cost.
|
|
412
|
+
|
|
413
|
+
`--crop <filter>` resolves a DOM element by the same filter syntax as `dom --filter`, then crops the screenshot to its bounding box before encoding. One tile (~1.6k vision tokens) instead of twelve (~19k) in the best case. If the filter matches nothing a warning is emitted to stderr and the full window is captured instead. Combines naturally with `--scale`.
|
|
241
414
|
|
|
242
415
|
### `scene`
|
|
243
416
|
|
|
@@ -248,14 +421,16 @@ agent-view scene # Full scene graph
|
|
|
248
421
|
agent-view scene --diff # Changes since last call
|
|
249
422
|
agent-view scene --filter "player" # Filter by name/type
|
|
250
423
|
agent-view scene --verbose # Extended props (alpha, scale, bounds)
|
|
424
|
+
agent-view scene --compact # Merge single-child chains onto one line
|
|
251
425
|
```
|
|
252
426
|
|
|
253
427
|
### `snap`
|
|
254
428
|
|
|
255
|
-
Combined DOM + scene graph in one call. Shows DOM always; scene section appears when a WebGL engine is detected.
|
|
429
|
+
Combined DOM + scene graph in one call. Shows DOM always; scene section appears when a WebGL engine is detected. Pass `--scale` to also capture a screenshot and append it as a third section.
|
|
256
430
|
|
|
257
431
|
```bash
|
|
258
432
|
agent-view snap
|
|
433
|
+
agent-view snap --scale 0.5 # DOM + Scene + Screenshot (path written to tmp)
|
|
259
434
|
```
|
|
260
435
|
|
|
261
436
|
### `wait`
|
|
@@ -301,85 +476,59 @@ Streams or dumps console output (`Runtime.consoleAPICalled` + `Log.entryAdded`)
|
|
|
301
476
|
```bash
|
|
302
477
|
agent-view console # buffered messages since attach
|
|
303
478
|
agent-view console --follow --timeout 10 # stream for 10s
|
|
304
|
-
agent-view console --
|
|
479
|
+
agent-view console --follow --until "ready" # exit as soon as a message contains "ready"
|
|
480
|
+
agent-view console --follow --until "/error/i" # exit on regex match (case-insensitive)
|
|
481
|
+
agent-view console --target IJ56KL # restrict to one target (exact id)
|
|
482
|
+
agent-view console --target sync-worker # restrict to one target (title/URL substring)
|
|
305
483
|
agent-view console --level error,warn # level filter
|
|
306
484
|
agent-view console --since "2026-04-26T10:00:00Z"
|
|
307
485
|
agent-view console --clear # drop in-memory ring
|
|
308
486
|
```
|
|
309
487
|
|
|
310
|
-
|
|
488
|
+
`--until <pattern>` requires `--follow`. Exits as soon as a message matches the pattern (substring or `/regex/flags`). On timeout without match exits non-zero with `Timeout: pattern not seen in <N>s`.
|
|
311
489
|
|
|
312
|
-
|
|
490
|
+
`--target` resolves the same way as `eval --target`: exact id wins, then title substring, then URL substring. If no match is found, an error is returned.
|
|
313
491
|
|
|
314
|
-
|
|
492
|
+
Default attached target types: `page`, `shared_worker`, `service_worker`. Override with `consoleTargets` in config.
|
|
315
493
|
|
|
316
|
-
|
|
494
|
+
### `watch`
|
|
317
495
|
|
|
318
|
-
|
|
496
|
+
Polls a JS expression and streams JSON-patch (RFC 6902) diffs as it changes. Closes the "what changed between click and final state?" gap that screenshots and DOM dumps can't cover. **Requires `"allowEval": true`** (same gate as `eval`).
|
|
319
497
|
|
|
320
498
|
```bash
|
|
321
|
-
agent-view
|
|
322
|
-
agent-view
|
|
499
|
+
agent-view watch "store.cart.total" # 250ms poll, exits at 10 changes or 30s
|
|
500
|
+
agent-view watch "appState" --interval 100 --duration 60 # tighter cadence, longer window
|
|
501
|
+
agent-view watch "store.status" --until "store.status === 'ready'" # wait-for assertion
|
|
502
|
+
agent-view watch "appState" --max-changes 1 # snapshot first change after a click
|
|
503
|
+
agent-view watch "appState" --json # NDJSON, one frame per line
|
|
323
504
|
```
|
|
324
505
|
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
| Command | Format | Why |
|
|
328
|
-
|---------|--------|-----|
|
|
329
|
-
| `dom`, `scene`, `snap` | Plain text | LLM-friendly, minimal tokens |
|
|
330
|
-
| `discover` | JSON | Machine-parseable |
|
|
331
|
-
| `screenshot` | File path | Agent reads the image |
|
|
332
|
-
|
|
333
|
-
## Claude Code plugin
|
|
334
|
-
|
|
335
|
-
agent-view ships as a Claude Code plugin with a built-in `verify` skill. The skill teaches Claude how to use agent-view for UI verification after code changes.
|
|
336
|
-
|
|
337
|
-
### Local development
|
|
506
|
+
Output frames: `init` (baseline value), `diff` (RFC 6902 ops since last frame), `error`, `stop`. SIGINT exits cleanly. Snapshot size cap 256 KB — narrow the expression (e.g. `store.cart.items.length`) when watching large objects.
|
|
338
507
|
|
|
339
|
-
|
|
340
|
-
claude --plugin-dir /path/to/agent-view
|
|
341
|
-
```
|
|
342
|
-
|
|
343
|
-
The skill becomes available as `/agent-view:verify`.
|
|
508
|
+
### `stop`
|
|
344
509
|
|
|
345
|
-
|
|
510
|
+
Stops the background lazy server.
|
|
346
511
|
|
|
347
|
-
|
|
348
|
-
# Find where the package is installed
|
|
349
|
-
npm root -g
|
|
350
|
-
# Use that path with --plugin-dir
|
|
351
|
-
claude --plugin-dir "$(npm root -g)/agent-view"
|
|
352
|
-
```
|
|
512
|
+
## Performance
|
|
353
513
|
|
|
354
|
-
|
|
514
|
+
Built for tight `dom → click → dom` loops. Typical Electron app, ~200 AX nodes:
|
|
355
515
|
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
agent-view dom --filter "login"
|
|
362
|
-
# RootWebArea "My App"
|
|
363
|
-
# textbox "Email" [ref=3]
|
|
364
|
-
# textbox "Password" [ref=5]
|
|
365
|
-
# button "Sign in" [ref=7]
|
|
366
|
-
|
|
367
|
-
# Fill credentials and submit
|
|
368
|
-
agent-view fill 3 "admin@example.com"
|
|
369
|
-
agent-view fill 5 "password123"
|
|
370
|
-
agent-view click 7
|
|
516
|
+
| Scenario | agent-view | Playwright (estimate) |
|
|
517
|
+
|--------------------------------|------------|-----------------------|
|
|
518
|
+
| `dom` cold fetch | 2ms | ~30–80ms |
|
|
519
|
+
| `dom` warm (cache hit) | 1ms | ~30–80ms |
|
|
520
|
+
| Full cycle `dom → click → dom` | 17ms | ~75ms |
|
|
371
521
|
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
```
|
|
522
|
+
What makes it fast: 300ms AX-tree cache (invalidated on `click`/`fill`/navigation; cached responses prefixed with
|
|
523
|
+
`[cache]`), parallel CDP calls in `click`, `Accessibility.queryAXTree` for filter lookups, and a single persistent CDP
|
|
524
|
+
WebSocket reused across commands (no relay).
|
|
376
525
|
|
|
377
526
|
## Troubleshooting
|
|
378
527
|
|
|
379
528
|
### CDP not responding
|
|
380
529
|
|
|
381
|
-
1. Check the port is listening: `curl -s http://localhost:
|
|
382
|
-
2. For electron-vite: make sure you use `--` before the flag: `npx electron-vite dev -- --remote-debugging-port=
|
|
530
|
+
1. Check the port is listening: `curl -s http://localhost:9876/json/version`
|
|
531
|
+
2. For electron-vite: make sure you use `--` before the flag: `npx electron-vite dev -- --remote-debugging-port=9876`
|
|
383
532
|
3. Restart the app — HMR doesn't restart the main process
|
|
384
533
|
|
|
385
534
|
### Stale refs after HMR
|