effing-use 0.1.1 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -8,21 +8,23 @@ Full Chromium automation in **3 tools, 3.9 KB**. Same pages, same clicks, same s
8
8
 
9
9
  ## Install (Bun-only)
10
10
 
11
- Requires [Bun](https://bun.sh) 1.4+. Installs from npm in seconds — the tarball is ~16 kB ([`effing-use` v0.1.1](https://www.npmjs.com/package/effing-use)):
11
+ Requires [Bun](https://bun.sh) 1.4+. Installs from npm in seconds — the tarball is ~16 kB ([`effing-use` v0.2.1](https://www.npmjs.com/package/effing-use)):
12
12
 
13
13
  ```bash
14
14
  # No install needed — bunx fetches from npm on first run
15
- bunx effing-use # STDIO (single editor) — runs src/index.ts via bin
16
- bunx effing-use-http # HTTP on :3123 (/mcp) — shared across editors
17
- bunx playwright install chromium --only-shell # one-time Chromium download (~150 MB)
15
+ bunx --package effing-use effing-use # CLI (needs running server)
16
+ bunx --package effing-use effing-use-http # HTTP MCP on :3123 (/mcp) — shared across editors
17
+ bunx --package effing-use effing-use-stdio # STDIO MCP (single editor)
18
+ bunx playwright install chromium --only-shell # one-time Chromium download (~150 MB)
18
19
  ```
19
20
 
20
21
  Optional — install globally so `effing-use` is on your PATH:
21
22
 
22
23
  ```bash
23
24
  bun install -g effing-use
24
- effing-use # STDIO
25
- effing-use-http # HTTP on :3123 (/mcp)
25
+ effing-use --help # CLI
26
+ effing-use-http # HTTP MCP on :3123 (/mcp)
27
+ effing-use-stdio # STDIO MCP
26
28
  ```
27
29
 
28
30
  The `playwright` npm package ships the driver, **not** the browser — every user needs Playwright's version-pinned Chromium once. From source instead:
@@ -30,41 +32,85 @@ The `playwright` npm package ships the driver, **not** the browser — every use
30
32
  ```bash
31
33
  bun install
32
34
  bunx playwright install chromium --only-shell
33
- bun src/index.ts # STDIO
34
- bun src/http.ts # HTTP on :3123 (/mcp)
35
+ bun src/cli.ts --help # CLI (needs running server)
36
+ bun src/http.ts # HTTP MCP on :3123 (/mcp)
37
+ bun src/index.ts # STDIO MCP
35
38
  ```
36
39
 
37
40
  Re-run `bunx playwright install chromium --only-shell` whenever you bump the `playwright` dependency (each Playwright version pins its own browser build). Verify with `bunx playwright install --dry-run chromium` or check `~/.cache/ms-playwright/`.
38
41
 
39
42
  **Why Chromium is separate:** Playwright supports multiple browsers and updates its pinned builds every release, so the binary can't live inside the npm tarball (ours is 16 kB). Docker users skip this — Chromium is baked into the `mcr.microsoft.com/playwright` base image.
40
43
 
44
+ ### npm vs source
45
+
46
+ | Source | Command | When to use |
47
+ | ------------------------------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------ |
48
+ | **npm** (`bunx --package effing-use`) | `effing-use`, `effing-use-http`, `effing-use-stdio` | Recommended — always matches published `package.json` version, no clone needed |
49
+ | **source** (`bun src/*.ts`) | `bun src/cli.ts`, `bun src/http.ts`, `bun src/index.ts` | Local dev / unreleased changes |
50
+
51
+ Both expose the same 3 bins: `effing-use` = CLI (HTTP client), `effing-use-stdio` = MCP stdio, `effing-use-http` = MCP http. Don't mix them — `effing-use` alone is **not** an MCP server (it prints CLI help and exits, causing `Failed to parse message` / `MCP server has stopped`).
52
+
41
53
  ## Run it in 60 seconds (recommended path)
42
54
 
43
55
  **Step 1/3 — Start the server.** One server, every local editor:
44
56
 
45
57
  ```bash
58
+ # Docker (recommended — Chromium baked in, no local install)
59
+ docker compose up --build -d
60
+ curl http://localhost:3123/healthz # {"ok":true,"name":"effing-use"}
61
+
62
+ # Or without Docker
46
63
  bun install
47
64
  bunx playwright install chromium --only-shell
48
- docker compose up --build -d
49
- curl http://localhost:3123/healthz
65
+ bun src/http.ts # or: bunx --package effing-use effing-use-http
66
+ ```
67
+
68
+ The HTTP server binds `0.0.0.0:3123` inside Docker (so forwarded ports work) and sets `idleTimeout: 0` so the MCP SSE stream isn't killed after 10s of idle time. Plain HTTP on loopback is intentional — add TLS at the edge for remote use.
69
+
70
+ **Step 2/3 — Connect.** Pick one transport:
71
+
72
+ **HTTP (shared — recommended for multiple VS Code windows):**
73
+
74
+ ```json
75
+ // ~/.config/Code/User/mcp.json (global, all workspaces) or .vscode/mcp.json
76
+ {
77
+ "servers": {
78
+ "effing-use": { "type": "http", "url": "http://localhost:3123/mcp" }
79
+ }
80
+ }
50
81
  ```
51
82
 
52
- Point any VS Code instance at `http://localhost:3123/mcp` (see `.vscode/mcp.json` → `effing-use (http)`). Plain HTTP on loopback is intentional — add TLS at the edge for remote use.
83
+ One Docker/bun process serves every window. Use same `sessionId` to share tabs, different `sessionId` to isolate.
53
84
 
54
- **Step 2/3 — Connect.** STDIO for one editor, HTTP for all of them:
85
+ **STDIO (isolated — one browser per window):**
55
86
 
56
87
  ```json
57
88
  {
58
- "mcpServers": {
89
+ "servers": {
59
90
  "effing-use": {
91
+ "type": "stdio",
60
92
  "command": "bunx",
61
- "args": ["effing-use"]
93
+ "args": ["--package", "effing-use", "effing-use-stdio"],
94
+ "env": {
95
+ "BROWSER_HEADLESS": "true",
96
+ "OUTPUT_DIR": "${workspaceFolder}/.browser-use"
97
+ }
62
98
  }
63
99
  }
64
100
  }
65
101
  ```
66
102
 
67
- Or HTTP: `http://localhost:3123/mcp` (via `bunx effing-use-http` or Docker). From source instead: `command: "bun"`, `args: ["/path/to/effing-use/src/index.ts"]`.
103
+ From source: `command: "bun"`, `args: ["/absolute/path/to/effing-use/src/index.ts"]`.
104
+
105
+ **CLI vs MCP — when to use which:**
106
+
107
+ | Surface | Command | Needs server? | Best for |
108
+ | --------------- | --------------------------------------- | ------------------------------- | ------------------------------- |
109
+ | **MCP (stdio)** | `effing-use-stdio` / `bun src/index.ts` | No (spawns own browser) | Single VS Code / Claude Desktop |
110
+ | **MCP (http)** | `effing-use-http` / `bun src/http.ts` | Yes (`:3123`) | Shared across editors, Docker |
111
+ | **CLI** | `effing-use` / `bun src/cli.ts` | Yes (`:3123`, `EFFING_USE_URL`) | Terminal agents, scripts, CI |
112
+
113
+ The CLI is a thin HTTP client over the same engine — `effing-use observe --kind snapshot --mode delta` and `browser_observe kind=snapshot mode=delta` hit the same code. Start the server once (`bun src/http.ts` or `docker compose up`), then use either face.
68
114
 
69
115
  **Step 3/3 — Drive.** Search Wikipedia in one call instead of two round-trips:
70
116
 
@@ -79,30 +125,102 @@ Or HTTP: `http://localhost:3123/mcp` (via `bunx effing-use-http` or Docker). Fro
79
125
  }
80
126
  ```
81
127
 
82
- No Docker? `bun src/index.ts` (STDIO) or `bun src/http.ts` (`:3123` `/mcp`) works directly.
128
+ No Docker? `bun src/cli.ts --help` (CLI), `bun src/index.ts` (STDIO MCP), or `bun src/http.ts` (`:3123` `/mcp`) works directly.
129
+
130
+ ### CLI usage
131
+
132
+ The CLI needs a running HTTP server (`EFFING_USE_URL` defaults to `http://localhost:3123/mcp`):
133
+
134
+ ```bash
135
+ # Observe
136
+ effing-use observe --kind snapshot --mode delta --session dev
137
+ effing-use observe --kind title --session dev
138
+ effing-use observe --kind screenshot --session dev
139
+
140
+ # Act
141
+ effing-use act --action open --value https://example.com --session dev
142
+ effing-use act --action click --target e5 --expect 'url~/dashboard' --session dev
143
+ effing-use act --action batch --file steps.json --session dev
144
+
145
+ # Extract
146
+ effing-use extract --kind text --selector main --session dev
147
+ effing-use extract --kind state --session dev
148
+
149
+ # With custom server URL
150
+ EFFING_USE_URL=http://localhost:3123/mcp effing-use observe --kind snapshot
151
+ ```
152
+
153
+ From npm without global install: `bunx --package effing-use effing-use observe --kind snapshot`.
154
+
155
+ ### Docker
156
+
157
+ ```bash
158
+ docker compose up --build -d # build + start (0.0.0.0:3123, idleTimeout:0)
159
+ docker compose logs -f effing-use # tail logs
160
+ curl http://localhost:3123/healthz # health check
161
+ docker compose down # stop (add -v to remove volumes)
162
+ # Rebuild after pulling new code
163
+ docker compose up --build -d
164
+ # Orphaned old container holding :3123? (renamed service)
165
+ docker rm -f effing-use-computer-use-1 && docker compose up -d
166
+ ```
167
+
168
+ Compose details: `restart: unless-stopped`, `init: true` + `ipc: host` (Playwright flags), `.browser-use/` bind-mounted, `EFFING_PORT` overrides host port (`EFFING_PORT=4000 docker compose up -d`).
83
169
 
84
170
  ## Why agents prefer 3 tools
85
171
 
86
- - 🪶 **Tiny handshake, full surface** — `browser_act` (27 actions), `browser_observe` (8 kinds), `browser_extract` (7 kinds). No schema bloat, no guessing which of 24 tools to call.
172
+ - 🪶 **Tiny handshake, full surface** — `browser_act` (32 actions), `browser_observe` (8 kinds + `mode`/`scope`), `browser_extract` (8 kinds incl. `state`). No schema bloat, no guessing which of 24 tools to call.
87
173
  - 🧠 **Context-safe by default** — snapshots capped at `OUTPUT_MAX_CHARS` (default 4000); full YAML/text saved under `.browser-use/`, never dumped inline. HN snapshot: 4,034-char preview, full file on disk.
88
174
  - 🖼️ **File paths, not base64** — screenshots, PDFs, traces return paths like `.browser-use/shot-*.png`. Read the file only when needed.
89
175
  - ⚡ **One call, not five** — `batch` runs fill+press flows in one turn (max 20 steps, stops on first error). `goal` plans or returns `E_GOAL_UNCLEAR` + `suggestedSteps` instead of hallucinating.
90
176
  - 🐳 **Shared, not spawned** — one Docker server serves every local VS Code instance. No per-client `npx` spawn.
91
177
 
178
+ ## Harness — verify, delta, state, replay
179
+
180
+ v2 turns the browser into a harness: AI actions are verifiable, diffable, and replayable — not just fire-and-forget clicks.
181
+
182
+ **1. Verify — fingerprints + expectations + failure contract**
183
+
184
+ - **Fingerprint registry** (`src/browser/identity.ts`): every `eN` ref is fingerprinted (role, accessibleName, textHash, box, pathHash). Stale refs rebind only on an UNAMBIGUOUS identity match (`rebound:true`); if several elements match equally they fail with `E_STALE` + hint — the engine never guesses a target.
185
+ - **Expect mini-language** (`expect` on any `browser_act`): `url~/dashboard` | `text~/Saved/` | `visible=.modal` | `gone=.spinner` — evaluated server-side, returns `E_EXPECT` / `E_BAD_EXPECT` on mismatch instead of hallucinated success. ReDoS-capped and regex-validated.
186
+ - **Failure contract**: after an uncertain mutation the engine sets `mustObserve:true` — next mutation fails with `E_MUST_OBSERVE` until you re-snapshot. No blind chains.
187
+ - **Evidence envelope**: every `browser_act` returns `effect: { urlChanged, urlBefore/After, domChanged, consoleErrors, networkFailures }` capped at `EFFECT_MAX_CHARS` (800) so the agent sees what actually happened.
188
+
189
+ **2. Delta — pay only for what changed**
190
+
191
+ - `browser_observe kind=snapshot mode=delta` (default) — MutationObserver dirty flag + baseline diff. Returns only `[changed]` lines or `unchanged:true` on stable pages (~90% token saving). `mode=full` for complete dump, `scope="<css>"` for subtree.
192
+ - `DELTA_DEFAULT=true` — flip to `false` to default to full snapshots.
193
+
194
+ **3. State — per-session memory**
195
+
196
+ - `browser_act action=note value="..."` appends to `notes` (capped `STATE_MAX_LINES=40`), `browser_extract kind=state` reads `notes` + `lastActions` ring (last 10). Persisted to `.browser-use/state/<session>.md` so agents survive context compaction.
197
+
198
+ **4. Record → Compile → Replay — deterministic macros**
199
+
200
+ - `record_start` / `record_stop` captures every step with resolved selectors + fingerprints. Secrets auto-redacted (`RECORD_REDACT=true`, `«redacted»` for password/otp/token fields).
201
+ - `compile` generates `.browser-use/macros/<name>.{ts,md}` — a Playwright `run(page)` function + a `SKILL.md` doc. Irreversible steps (`submit`/`pay`/`delete`/etc.) are flagged `requiresApproval`.
202
+ - `replay` replays deterministically; pauses with `E_APPROVAL_REQUIRED` until `approve:true` if any irreversible step exists.
203
+
204
+ **5. CLI — same engine, no MCP client**
205
+
206
+ - `effing-use observe/act/extract` over `EFFING_USE_URL` (`http://localhost:3123/mcp`) — for terminal agents, scripts, and CI. See [CLI usage](#cli-usage).
207
+
92
208
  ## The loop (agents: follow this order)
93
209
 
94
- 1. `browser_observe` kind=`snapshot` → get `[eN]` refs (never guess refs, re-snapshot after navigation)
95
- 2. `browser_act` to interact — prefer `batch` with `steps[]`
96
- 3. `browser_observe` kind=`screenshot` → verify visually (you get a path)
97
- 4. `browser_extract` kind=`text`|`table`|`query` → scrape structured data
210
+ 1. `browser_observe` kind=`snapshot` → get `[eN]` refs (default `mode: delta` — only changed lines; `mode: full` for complete dump; `scope: "<css>"` for subtree). Never guess refs, re-snapshot after navigation.
211
+ 2. `browser_act` to interact — prefer `batch` with `steps[]`; add `expect: "url~/dashboard"` for deterministic post-conditions
212
+ 3. `browser_observe` kind=`snapshot` again — delta returns `unchanged:true` or `[changed]` lines
213
+ 4. `browser_extract` kind=`text`|`table`|`query`|`state` → scrape or read task state (`notes` + `lastActions`)
214
+
215
+ CLI equivalent: `effing-use observe --kind snapshot --mode delta` / `effing-use act --action click --target e5 --expect 'url~/dashboard'` / `effing-use extract --kind state`
98
216
 
99
217
  ## Tools
100
218
 
101
- - `browser_act` — open/goto, click, dblclick, fill, type, press, select, check/uncheck, hover, drag, upload, scroll, back/forward/reload, wait, dialog_accept/dismiss, tabs (new/select/close), resize, close, `goal`, `batch`
102
- - `browser_observe` — snapshot (e-refs), screenshot (path), url, title, console (last N), network (method/url/status ring), tabs, focused
103
- - `browser_extract` — text, html, table (≤100 rows JSON), query (text|href|json), pdf, trace_start/stop
219
+ - `browser_act` — open/goto, click, dblclick, fill, type, press, select, check/uncheck, hover, drag, upload, scroll, back/forward/reload, wait, dialog_accept/dismiss, tabs (new/select/close), resize, close, `goal`, `batch`, `note`, `record_start`/`record_stop`, `compile`, `replay` (+ `expect` + `approve`)
220
+ - `browser_observe` — snapshot (e-refs, `mode: full|delta` default delta, `scope`), screenshot (path), url, title, console (last N), network (method/url/status ring), tabs, focused
221
+ - `browser_extract` — text, html, table (≤100 rows JSON), query (text|href|json), pdf, trace_start/stop, `state`
104
222
 
105
- Errors are always `{ ok: false, code, message, hint }` with `E_NOT_FOUND | E_TIMEOUT | E_NO_PAGE | E_BAD_INPUT | E_GOAL_UNCLEAR` — never a stack trace.
223
+ Errors are always `{ ok: false, code, message, hint }` with `E_NOT_FOUND | E_TIMEOUT | E_NO_PAGE | E_BAD_INPUT | E_GOAL_UNCLEAR | E_STALE | E_EXPECT | E_BAD_EXPECT | E_MUST_OBSERVE | E_APPROVAL_REQUIRED` — never a stack trace.
106
224
 
107
225
  ## effing-use vs Playwright MCP
108
226
 
@@ -115,9 +233,30 @@ Errors are always `{ ok: false, code, message, hint }` with `E_NOT_FOUND | E_TIM
115
233
 
116
234
  Need Firefox/WebKit, device emulation, or persistent profiles? Use Playwright MCP. Need context budget for Chromium work? Stay here. Full breakdown in [`docs/COMPARISON.md`](docs/COMPARISON.md).
117
235
 
236
+ ## Record → replay (harness)
237
+
238
+ ```bash
239
+ # MCP — capture any flow, compile to code + skill, replay deterministically
240
+ browser_act action=record_start value=my-flow
241
+ # ... do the flow (clicks, fills, etc.) ...
242
+ browser_act action=record_stop # -> .browser-use/recordings/my-flow.json (secrets redacted)
243
+ browser_act action=compile value=my-flow # -> .browser-use/macros/my-flow.{ts,md}
244
+ browser_act action=replay value=my-flow # pauses with E_APPROVAL_REQUIRED if irreversible
245
+ browser_act action=replay value=my-flow approve=true # replay with approval
246
+
247
+ # CLI — same flow over HTTP
248
+ EFFING_USE_URL=http://localhost:3123/mcp effing-use act --action record_start --value my-flow
249
+ # ... do the flow via CLI or MCP ...
250
+ EFFING_USE_URL=http://localhost:3123/mcp effing-use act --action record_stop
251
+ EFFING_USE_URL=http://localhost:3123/mcp effing-use act --action compile --value my-flow
252
+ EFFING_USE_URL=http://localhost:3123/mcp effing-use act --action replay --value my-flow --approve true
253
+ ```
254
+
255
+ Artifacts: `recordings/*.json` (raw steps), `macros/*.ts` (Playwright `run(page)`), `macros/*.md` (skill doc). Redaction and approval gates are on by default (`RECORD_REDACT=true`).
256
+
118
257
  ## Config
119
258
 
120
- Env defaults in [`.env.example`](.env.example): `BROWSER_HEADLESS`, `BROWSER_VIEWPORT_W/H`, `BROWSER_TIMEOUT_MS`, `OUTPUT_DIR` (`.browser-use/`, gitignored), `OUTPUT_MAX_CHARS`, `ALLOW_EVAL`.
259
+ Env defaults in [`.env.example`](.env.example): `BROWSER_HEADLESS`, `BROWSER_VIEWPORT_W/H`, `BROWSER_TIMEOUT_MS`, `OUTPUT_DIR` (`.browser-use/`, gitignored), `OUTPUT_MAX_CHARS`, `ALLOW_EVAL`, `DELTA_DEFAULT`, `EFFECT_MAX_CHARS`, `STATE_MAX_LINES`, `RECORD_REDACT`, `EFFING_USE_URL`.
121
260
 
122
261
  Agent skill: `skills/effing-use/SKILL.md` (skills.sh-ready).
123
262
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "effing-use",
3
- "version": "0.1.1",
3
+ "version": "0.2.1",
4
4
  "description": "Token-efficient browser control: 3 tools (act, observe, extract) built with tmcp + Bun + Playwright",
5
5
  "license": "MIT",
6
6
  "author": "Michael Obubelebra Amachree",
@@ -12,8 +12,9 @@
12
12
  "type": "module",
13
13
  "main": "src/index.ts",
14
14
  "bin": {
15
- "effing-use": "src/index.ts",
16
- "effing-use-http": "src/http.ts"
15
+ "effing-use": "src/cli.ts",
16
+ "effing-use-http": "src/http.ts",
17
+ "effing-use-stdio": "src/index.ts"
17
18
  },
18
19
  "files": [
19
20
  "src",
@@ -15,40 +15,67 @@ cost than 21-tool browser servers. `tools/list` is ~3.9KB.
15
15
 
16
16
  ## The loop (always follow this order)
17
17
 
18
- 1. `browser_observe` with `kind: "snapshot"` → get `[eN]` refs.
19
- 2. `browser_act` to interact (`open`, `click`, `fill`, `type`, `press`, `select`, `check`, `wait`, …).
20
- 3. `browser_observe` with `kind: "screenshot"` → verify visually (returns a file path).
21
- 4. `browser_extract` with `kind: "text" | "table" | "query"` → scrape.
18
+ 1. `browser_observe` with `kind: "snapshot"` → get `[eN]` refs. Default is `mode: "delta"` (`[changed]`/`[removed]` lines since last observe; `unchanged:true` when the page is quiet — navigation automatically forces a fresh `mode: "full"`). Use `mode: "full"` for a complete dump, `scope: "<css>"` to observe a subtree.
19
+ 2. `browser_act` to interact (`open`, `click`, `fill`, `type`, `press`, `select`, `check`, `wait`, …). Add `expect: "url~/dashboard"` or `text~/Saved/` for deterministic post-conditions.
20
+ 3. `browser_observe` with `kind: "snapshot"` again — delta returns `unchanged:true` if nothing changed, or `[changed]`/`[removed]` lines.
21
+ 4. `browser_extract` with `kind: "text" | "table" | "query" | "state"` → scrape or read task state.
22
22
 
23
23
  Rules:
24
24
 
25
- - Never guess refs. Re-snapshot after every navigation.
25
+ - Never guess refs. Re-snapshot after every navigation (refs are page-keyed — a cross-page stale ref returns `E_NOT_FOUND`). On a same-page re-render the engine rebinds only an UNAMBIGUOUS identity match (result carries `rebound:true`); if the fingerprint matches several elements it fails with `E_STALE` instead of guessing — re-observe.
26
26
  - Prefer `batch`: one `browser_act` with `steps[]` for fill+press flows (max 20 steps, stops on first error).
27
27
  - Large outputs are files under `.browser-use/` (gitignored). Read the path, not the preview.
28
28
  - Snapshots are capped at `OUTPUT_MAX_CHARS` (default 4000) with `…[truncated N chars, see file]`.
29
+ - After an uncertain mutation the engine sets `mustObserve:true` — next mutation fails with `E_MUST_OBSERVE` until you observe.
29
30
 
30
31
  ## Tool cheat sheet
31
32
 
32
- **browser_act** — `action` + optional `target` (e-ref, `role=` selector, or CSS) + `value`:
33
+ **browser_act** — `action` + optional `target` (e-ref, `role=` selector, or CSS) + `value` + `expect` + `approve`:
33
34
  `open`/`goto` (URL in `value`), `click`, `dblclick`, `fill`, `type`,
34
35
  `press` (key like `Enter`), `select`, `check`/`uncheck`, `hover`,
35
36
  `drag` (start in `target`, end in `value`), `upload` (comma-separated paths in `value`),
36
37
  `scroll` (`up`/`down`/`top`/`bottom` or a target), `back`/`forward`/`reload`,
37
38
  `wait` (`ms:500`, `text:Saved`, or a ref), `dialog_accept`/`dialog_dismiss` (arm before the triggering step),
38
39
  `resize` (`1280x800` in `value`), `tab_new`/`tab_select`/`tab_close`, `close`,
39
- `goal` (deterministic add-todo/search planner, else `E_GOAL_UNCLEAR` + `suggestedSteps`),
40
- `batch` (needs `steps[]`).
40
+ `goal` (deterministic add-todo/search planner, else `E_GOAL_UNCLEAR` + `suggestedSteps`; counts as a mutation — subject to the `mustObserve` guard, evidence, and recording),
41
+ `batch` (needs `steps[]`), `note` (append to task state), `record_start`/`record_stop` (capture flow), `compile` (standalone `.ts` + SKILL.md; selectors resolved id → name → ARIA → data-\* → placeholder → text), `replay` (deterministic replay; runs benign steps, halts AT approval-gated steps).
42
+ `expect` mini-language: `url~<regex>` | `text~<regex>` | `visible=<css>` | `gone=<css>` — evaluated in code, returns `E_EXPECT` or `E_BAD_EXPECT`.
41
43
 
42
- **browser_observe** — read-only: `snapshot` (e-refs), `screenshot` (file path,
44
+ **browser_observe** — read-only: `snapshot` (e-refs, `mode: full|delta` default delta, `scope: <css>`), `screenshot` (file path,
43
45
  `full` page by default), `url`, `title`, `console` (last N, `limit`),
44
46
  `network` (method/url/status ring), `tabs`, `focused` (activeElement HTML).
45
47
 
46
48
  **browser_extract** — `text`, `html`, `table` (≤100 rows as JSON),
47
49
  `query` (`selector` + `mode: text|href|json`), `pdf` (headless Chromium only),
48
- `trace_start`/`trace_stop` (Playwright trace zip).
50
+ `trace_start`/`trace_stop` (Playwright trace zip), `state` (task notes + lastActions ring).
49
51
 
50
52
  Errors always come back as `{ ok: false, code, message, hint }` with codes
51
- `E_NOT_FOUND | E_TIMEOUT | E_NO_PAGE | E_BAD_INPUT | E_GOAL_UNCLEAR` — never a stack trace.
53
+ `E_NOT_FOUND | E_TIMEOUT | E_NO_PAGE | E_BAD_INPUT | E_GOAL_UNCLEAR | E_STALE | E_EXPECT | E_BAD_EXPECT | E_MUST_OBSERVE | E_APPROVAL_REQUIRED` — never a stack trace.
54
+
55
+ ## CLI face (same engine, no MCP client needed)
56
+
57
+ ```bash
58
+ effing-use observe --kind snapshot --mode delta --session dev
59
+ effing-use act --action click --target e5 --expect 'url~/dashboard' --session dev
60
+ effing-use extract --kind state --session dev
61
+ ```
62
+
63
+ Set `EFFING_USE_URL` (default `http://localhost:3123/mcp`). Requires a running `bun src/http.ts` server.
64
+
65
+ ## Record → replay
66
+
67
+ ```bash
68
+ # via MCP
69
+ browser_act action=record_start value=my-flow
70
+ # ... do the flow ...
71
+ browser_act action=record_stop
72
+ browser_act action=compile value=my-flow # -> .browser-use/macros/my-flow.{ts,md}
73
+ browser_act action=replay value=my-flow # runs benign steps, halts AT the first
74
+ # approval-gated step → E_APPROVAL_REQUIRED
75
+ # { step: N (1-indexed), completed: C }
76
+ browser_act action=replay value=my-flow approve=true # resumes at the gate; the
77
+ # benign prefix is NOT re-run (cursor)
78
+ ```
52
79
 
53
80
  ## Setup
54
81
 
@@ -0,0 +1,173 @@
1
+ import type { Page } from "playwright";
2
+
3
+ type PageState = {
4
+ dirty: boolean;
5
+ baseline: string | null;
6
+ baselineUrl: string | null;
7
+ };
8
+
9
+ const pageStates = new Map<string, PageState>();
10
+
11
+ function key(sessionId: string): string {
12
+ return sessionId;
13
+ }
14
+
15
+ export function ensureState(sessionId: string): PageState {
16
+ let s = pageStates.get(key(sessionId));
17
+ if (!s) {
18
+ s = { dirty: true, baseline: null, baselineUrl: null };
19
+ pageStates.set(key(sessionId), s);
20
+ }
21
+ return s;
22
+ }
23
+
24
+ export function markDirty(sessionId: string): void {
25
+ ensureState(sessionId).dirty = true;
26
+ }
27
+
28
+ export function markClean(sessionId: string): void {
29
+ ensureState(sessionId).dirty = false;
30
+ }
31
+
32
+ export function setBaseline(
33
+ sessionId: string,
34
+ snapshot: string,
35
+ url: string,
36
+ ): void {
37
+ const s = ensureState(sessionId);
38
+ s.baseline = snapshot;
39
+ s.baselineUrl = url;
40
+ s.dirty = false;
41
+ }
42
+
43
+ export function getBaseline(sessionId: string): string | null {
44
+ return ensureState(sessionId).baseline;
45
+ }
46
+
47
+ /** URL the current baseline was captured on — forces mode:full after navigation (plan §5.1). */
48
+ export function getBaselineUrl(sessionId: string): string | null {
49
+ return pageStates.get(key(sessionId))?.baselineUrl ?? null;
50
+ }
51
+
52
+ export function isDirty(sessionId: string): boolean {
53
+ return ensureState(sessionId).dirty;
54
+ }
55
+
56
+ export function clearDelta(sessionId: string): void {
57
+ pageStates.delete(key(sessionId));
58
+ }
59
+
60
+ export async function injectDirtyObserver(
61
+ page: Page,
62
+ sessionId: string,
63
+ ): Promise<void> {
64
+ try {
65
+ await page.evaluate((sid) => {
66
+ const w = window as unknown as {
67
+ __effDirty?: boolean;
68
+ __effSid?: string;
69
+ __effObs?: MutationObserver;
70
+ };
71
+ w.__effDirty = false;
72
+ w.__effSid = sid;
73
+ if (w.__effObs) w.__effObs.disconnect();
74
+ // Attribute spam that never changes the snapshot's ref lines (class,
75
+ // style, expand/animation state) used to mark every live SPA permanently
76
+ // dirty — the cheap "unchanged" fast path never fired. Ignore those;
77
+ // content changes still arrive via childList/characterData/input/change.
78
+ const NOISE_ATTRS = [
79
+ "class",
80
+ "style",
81
+ "aria-expanded",
82
+ "data-state",
83
+ "data-orientation",
84
+ "data-scroll-state",
85
+ "data-highlighted",
86
+ "data-hovered",
87
+ "data-dragging",
88
+ "data-resizing",
89
+ "data-index",
90
+ ];
91
+ const obs = new MutationObserver((muts) => {
92
+ for (const m of muts) {
93
+ if (m.type === "attributes") {
94
+ const name = m.attributeName || "";
95
+ if (NOISE_ATTRS.indexOf(name) !== -1) continue;
96
+ }
97
+ w.__effDirty = true;
98
+ return;
99
+ }
100
+ });
101
+ obs.observe(document.documentElement, {
102
+ childList: true,
103
+ subtree: true,
104
+ attributes: true,
105
+ characterData: true,
106
+ });
107
+ document.addEventListener(
108
+ "input",
109
+ () => {
110
+ w.__effDirty = true;
111
+ },
112
+ true,
113
+ );
114
+ document.addEventListener(
115
+ "change",
116
+ () => {
117
+ w.__effDirty = true;
118
+ },
119
+ true,
120
+ );
121
+ w.__effObs = obs;
122
+ }, sessionId);
123
+ } catch {
124
+ // ignore injection failures (e.g. page not ready)
125
+ }
126
+ }
127
+
128
+ export async function checkDirty(page: Page): Promise<boolean> {
129
+ try {
130
+ const d = await page.evaluate(
131
+ () => (window as unknown as { __effDirty?: boolean }).__effDirty,
132
+ );
133
+ return Boolean(d);
134
+ } catch {
135
+ return true;
136
+ }
137
+ }
138
+
139
+ export async function clearDirtyFlag(page: Page): Promise<void> {
140
+ try {
141
+ await page.evaluate(() => {
142
+ (window as unknown as { __effDirty?: boolean }).__effDirty = false;
143
+ });
144
+ } catch {
145
+ /* ignore */
146
+ }
147
+ }
148
+
149
+ export function computeDelta(
150
+ baseline: string | null,
151
+ current: string,
152
+ ): { delta: string; unchanged: boolean } {
153
+ if (!baseline) return { delta: current, unchanged: false };
154
+ if (baseline === current) return { delta: "", unchanged: true };
155
+ // Simple line diff: emit only changed lines with [changed] markers
156
+ const baseLines = baseline.split("\n");
157
+ const curLines = current.split("\n");
158
+ const baseSet = new Set(baseLines);
159
+ const curSet = new Set(curLines);
160
+ const changed = curLines.filter((l) => !baseSet.has(l));
161
+ const removed = baseLines.filter((l) => !curSet.has(l));
162
+ if (changed.length === 0 && removed.length === 0)
163
+ return { delta: "", unchanged: true };
164
+ const unchangedCount = curLines.length - changed.length;
165
+ const header = `…${unchangedCount} unchanged lines…`;
166
+ const delta = [
167
+ header,
168
+ ...changed.map((l) => `[changed] ${l}`),
169
+ // plan §5.1: removed nodes are reported explicitly
170
+ ...removed.map((l) => `[removed] ${l}`),
171
+ ].join("\n");
172
+ return { delta, unchanged: false };
173
+ }