effing-use 0.1.1 → 0.2.0

package/README.md

@@ -12,8 +12,9 @@ Requires [Bun](https://bun.sh) 1.4+. Installs from npm in seconds — the tarbal
 
 ```bash
 # No install needed — bunx fetches from npm on first run
-bunx effing-use               # STDIO (single editor) — runs src/index.ts via bin
-bunx effing-use-http          # HTTP on :3123 (/mcp) — shared across editors
+bunx effing-use               # CLI (needs running server) — effing-use observe/act/extract
+bunx effing-use-http          # HTTP MCP on :3123 (/mcp) — shared across editors
+bunx effing-use-stdio         # STDIO MCP (single editor) — legacy bin
 bunx playwright install chromium --only-shell   # one-time Chromium download (~150 MB)
 ```
 
@@ -21,8 +22,9 @@ Optional — install globally so `effing-use` is on your PATH:
 
 ```bash
 bun install -g effing-use
-effing-use                    # STDIO
-effing-use-http               # HTTP on :3123 (/mcp)
+effing-use --help             # CLI
+effing-use-http               # HTTP MCP on :3123 (/mcp)
+effing-use-stdio              # STDIO MCP
 ```
 
 The `playwright` npm package ships the driver, **not** the browser — every user needs Playwright's version-pinned Chromium once. From source instead:
@@ -30,8 +32,9 @@ The `playwright` npm package ships the driver, **not** the browser — every use
 ```bash
 bun install
 bunx playwright install chromium --only-shell
-bun src/index.ts              # STDIO
-bun src/http.ts               # HTTP on :3123 (/mcp)
+bun src/cli.ts --help         # CLI (needs running server)
+bun src/http.ts               # HTTP MCP on :3123 (/mcp)
+bun src/index.ts              # STDIO MCP
 ```
 
 Re-run `bunx playwright install chromium --only-shell` whenever you bump the `playwright` dependency (each Playwright version pins its own browser build). Verify with `bunx playwright install --dry-run chromium` or check `~/.cache/ms-playwright/`.
@@ -58,7 +61,7 @@ Point any VS Code instance at `http://localhost:3123/mcp` (see `.vscode/mcp.json
   "mcpServers": {
     "effing-use": {
       "command": "bunx",
-      "args": ["effing-use"]
+      "args": ["effing-use-stdio"]
     }
   }
 }
@@ -66,6 +69,16 @@ Point any VS Code instance at `http://localhost:3123/mcp` (see `.vscode/mcp.json
 
 Or HTTP: `http://localhost:3123/mcp` (via `bunx effing-use-http` or Docker). From source instead: `command: "bun"`, `args: ["/path/to/effing-use/src/index.ts"]`.
 
+**CLI vs MCP — when to use which:**
+
+| Surface | Command | Needs server? | Best for |
+|---------|---------|---------------|----------|
+| **MCP (stdio)** | `effing-use-stdio` / `bun src/index.ts` | No (spawns own browser) | Single VS Code / Claude Desktop |
+| **MCP (http)** | `effing-use-http` / `bun src/http.ts` | Yes (`:3123`) | Shared across editors, Docker |
+| **CLI** | `effing-use` / `bun src/cli.ts` | Yes (`:3123`, `EFFING_USE_URL`) | Terminal agents, scripts, CI |
+
+The CLI is a thin HTTP client over the same engine — `effing-use observe --kind snapshot --mode delta` and `browser_observe kind=snapshot mode=delta` hit the same code. Start the server once (`bun src/http.ts` or `docker compose up`), then use either face.
+
 **Step 3/3 — Drive.** Search Wikipedia in one call instead of two round-trips:
 
 ```jsonc
@@ -79,11 +92,11 @@ Or HTTP: `http://localhost:3123/mcp` (via `bunx effing-use-http` or Docker). Fro
 }
 ```
 
-No Docker? `bun src/index.ts` (STDIO) or `bun src/http.ts` (`:3123` `/mcp`) works directly.
+No Docker? `bun src/cli.ts --help` (CLI), `bun src/index.ts` (STDIO MCP), or `bun src/http.ts` (`:3123` `/mcp`) works directly.
 
 ## Why agents prefer 3 tools
 
-- 🪶 **Tiny handshake, full surface** — `browser_act` (27 actions), `browser_observe` (8 kinds), `browser_extract` (7 kinds). No schema bloat, no guessing which of 24 tools to call.
+- 🪶 **Tiny handshake, full surface** — `browser_act` (32 actions), `browser_observe` (8 kinds + `mode`/`scope`), `browser_extract` (8 kinds incl. `state`). No schema bloat, no guessing which of 24 tools to call.
 - 🧠 **Context-safe by default** — snapshots capped at `OUTPUT_MAX_CHARS` (default 4000); full YAML/text saved under `.browser-use/`, never dumped inline. HN snapshot: 4,034-char preview, full file on disk.
 - 🖼️ **File paths, not base64** — screenshots, PDFs, traces return paths like `.browser-use/shot-*.png`. Read the file only when needed.
 - ⚡ **One call, not five** — `batch` runs fill+press flows in one turn (max 20 steps, stops on first error). `goal` plans or returns `E_GOAL_UNCLEAR` + `suggestedSteps` instead of hallucinating.
@@ -91,18 +104,20 @@ No Docker? `bun src/index.ts` (STDIO) or `bun src/http.ts` (`:3123` `/mcp`) work
 
 ## The loop (agents: follow this order)
 
-1. `browser_observe` kind=`snapshot` → get `[eN]` refs (never guess refs, re-snapshot after navigation)
-2. `browser_act` to interact — prefer `batch` with `steps[]`
-3. `browser_observe` kind=`screenshot` → verify visually (you get a path)
-4. `browser_extract` kind=`text`|`table`|`query` → scrape structured data
+1. `browser_observe` kind=`snapshot` → get `[eN]` refs (default `mode: delta` — only changed lines; `mode: full` for complete dump; `scope: "<css>"` for subtree). Never guess refs, re-snapshot after navigation.
+2. `browser_act` to interact — prefer `batch` with `steps[]`; add `expect: "url~/dashboard"` for deterministic post-conditions
+3. `browser_observe` kind=`snapshot` again — delta returns `unchanged:true` or `[changed]` lines
+4. `browser_extract` kind=`text`|`table`|`query`|`state` → scrape or read task state (`notes` + `lastActions`)
+
+CLI equivalent: `effing-use observe --kind snapshot --mode delta` / `effing-use act --action click --target e5 --expect 'url~/dashboard'` / `effing-use extract --kind state`
 
 ## Tools
 
-- `browser_act` — open/goto, click, dblclick, fill, type, press, select, check/uncheck, hover, drag, upload, scroll, back/forward/reload, wait, dialog_accept/dismiss, tabs (new/select/close), resize, close, `goal`, `batch`
-- `browser_observe` — snapshot (e-refs), screenshot (path), url, title, console (last N), network (method/url/status ring), tabs, focused
-- `browser_extract` — text, html, table (≤100 rows JSON), query (text|href|json), pdf, trace_start/stop
+- `browser_act` — open/goto, click, dblclick, fill, type, press, select, check/uncheck, hover, drag, upload, scroll, back/forward/reload, wait, dialog_accept/dismiss, tabs (new/select/close), resize, close, `goal`, `batch`, `note`, `record_start`/`record_stop`, `compile`, `replay` (+ `expect` + `approve`)
+- `browser_observe` — snapshot (e-refs, `mode: full|delta` default delta, `scope`), screenshot (path), url, title, console (last N), network (method/url/status ring), tabs, focused
+- `browser_extract` — text, html, table (≤100 rows JSON), query (text|href|json), pdf, trace_start/stop, `state`
 
-Errors are always `{ ok: false, code, message, hint }` with `E_NOT_FOUND | E_TIMEOUT | E_NO_PAGE | E_BAD_INPUT | E_GOAL_UNCLEAR` — never a stack trace.
+Errors are always `{ ok: false, code, message, hint }` with `E_NOT_FOUND | E_TIMEOUT | E_NO_PAGE | E_BAD_INPUT | E_GOAL_UNCLEAR | E_STALE | E_EXPECT | E_BAD_EXPECT | E_MUST_OBSERVE | E_APPROVAL_REQUIRED` — never a stack trace.
 
 ## effing-use vs Playwright MCP
 
@@ -115,9 +130,21 @@ Errors are always `{ ok: false, code, message, hint }` with `E_NOT_FOUND | E_TIM
 
 Need Firefox/WebKit, device emulation, or persistent profiles? Use Playwright MCP. Need context budget for Chromium work? Stay here. Full breakdown in [`docs/COMPARISON.md`](docs/COMPARISON.md).
 
+## Record → replay
+
+```bash
+# MCP
+browser_act action=record_start value=my-flow
+# ... do the flow ...
+browser_act action=record_stop
+browser_act action=compile value=my-flow   # -> .browser-use/macros/my-flow.{ts,md}
+browser_act action=replay value=my-flow    # pauses with E_APPROVAL_REQUIRED if irreversible
+browser_act action=replay value=my-flow approve=true
+```
+
 ## Config
 
-Env defaults in [`.env.example`](.env.example): `BROWSER_HEADLESS`, `BROWSER_VIEWPORT_W/H`, `BROWSER_TIMEOUT_MS`, `OUTPUT_DIR` (`.browser-use/`, gitignored), `OUTPUT_MAX_CHARS`, `ALLOW_EVAL`.
+Env defaults in [`.env.example`](.env.example): `BROWSER_HEADLESS`, `BROWSER_VIEWPORT_W/H`, `BROWSER_TIMEOUT_MS`, `OUTPUT_DIR` (`.browser-use/`, gitignored), `OUTPUT_MAX_CHARS`, `ALLOW_EVAL`, `DELTA_DEFAULT`, `EFFECT_MAX_CHARS`, `STATE_MAX_LINES`, `RECORD_REDACT`, `EFFING_USE_URL`.
 
 Agent skill: `skills/effing-use/SKILL.md` (skills.sh-ready).
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "effing-use",
-  "version": "0.1.1",
+  "version": "0.2.0",
   "description": "Token-efficient browser control: 3 tools (act, observe, extract) built with tmcp + Bun + Playwright",
   "license": "MIT",
   "author": "Michael Obubelebra Amachree",
@@ -12,8 +12,9 @@
   "type": "module",
   "main": "src/index.ts",
   "bin": {
-    "effing-use": "src/index.ts",
-    "effing-use-http": "src/http.ts"
+    "effing-use": "src/cli.ts",
+    "effing-use-http": "src/http.ts",
+    "effing-use-stdio": "src/index.ts"
   },
   "files": [
     "src",

package/skills/effing-use/SKILL.md

@@ -15,21 +15,22 @@ cost than 21-tool browser servers. `tools/list` is ~3.9KB.
 
 ## The loop (always follow this order)
 
-1. `browser_observe` with `kind: "snapshot"` → get `[eN]` refs.
-2. `browser_act` to interact (`open`, `click`, `fill`, `type`, `press`, `select`, `check`, `wait`, …).
-3. `browser_observe` with `kind: "screenshot"` → verify visually (returns a file path).
-4. `browser_extract` with `kind: "text" | "table" | "query"` → scrape.
+1. `browser_observe` with `kind: "snapshot"` → get `[eN]` refs. Default is `mode: "delta"` (only changes since last observe); use `mode: "full"` for a complete dump. Use `scope: "<css>"` to observe a subtree.
+2. `browser_act` to interact (`open`, `click`, `fill`, `type`, `press`, `select`, `check`, `wait`, …). Add `expect: "url~/dashboard"` or `text~/Saved/` for deterministic post-conditions.
+3. `browser_observe` with `kind: "snapshot"` again — delta returns `unchanged:true` if nothing changed, or `[changed]` lines.
+4. `browser_extract` with `kind: "text" | "table" | "query" | "state"` → scrape or read task state.
 
 Rules:
 
-- Never guess refs. Re-snapshot after every navigation.
+- Never guess refs. Re-snapshot after every navigation. Stale refs return `E_STALE` or auto-rebind with `rebound:true`.
 - Prefer `batch`: one `browser_act` with `steps[]` for fill+press flows (max 20 steps, stops on first error).
 - Large outputs are files under `.browser-use/` (gitignored). Read the path, not the preview.
 - Snapshots are capped at `OUTPUT_MAX_CHARS` (default 4000) with `…[truncated N chars, see file]`.
+- After an uncertain mutation the engine sets `mustObserve:true` — next mutation fails with `E_MUST_OBSERVE` until you observe.
 
 ## Tool cheat sheet
 
-**browser_act** — `action` + optional `target` (e-ref, `role=` selector, or CSS) + `value`:
+**browser_act** — `action` + optional `target` (e-ref, `role=` selector, or CSS) + `value` + `expect` + `approve`:
 `open`/`goto` (URL in `value`), `click`, `dblclick`, `fill`, `type`,
 `press` (key like `Enter`), `select`, `check`/`uncheck`, `hover`,
 `drag` (start in `target`, end in `value`), `upload` (comma-separated paths in `value`),
@@ -37,18 +38,41 @@ Rules:
 `wait` (`ms:500`, `text:Saved`, or a ref), `dialog_accept`/`dialog_dismiss` (arm before the triggering step),
 `resize` (`1280x800` in `value`), `tab_new`/`tab_select`/`tab_close`, `close`,
 `goal` (deterministic add-todo/search planner, else `E_GOAL_UNCLEAR` + `suggestedSteps`),
-`batch` (needs `steps[]`).
+`batch` (needs `steps[]`), `note` (append to task state), `record_start`/`record_stop` (capture flow), `compile` (macro+SKILL.md), `replay` (deterministic replay, needs `approve:true` for irreversible steps).
+`expect` mini-language: `url~<regex>` | `text~<regex>` | `visible=<css>` | `gone=<css>` — evaluated in code, returns `E_EXPECT` or `E_BAD_EXPECT`.
 
-**browser_observe** — read-only: `snapshot` (e-refs), `screenshot` (file path,
+**browser_observe** — read-only: `snapshot` (e-refs, `mode: full|delta` default delta, `scope: <css>`), `screenshot` (file path,
 `full` page by default), `url`, `title`, `console` (last N, `limit`),
 `network` (method/url/status ring), `tabs`, `focused` (activeElement HTML).
 
 **browser_extract** — `text`, `html`, `table` (≤100 rows as JSON),
 `query` (`selector` + `mode: text|href|json`), `pdf` (headless Chromium only),
-`trace_start`/`trace_stop` (Playwright trace zip).
+`trace_start`/`trace_stop` (Playwright trace zip), `state` (task notes + lastActions ring).
 
 Errors always come back as `{ ok: false, code, message, hint }` with codes
-`E_NOT_FOUND | E_TIMEOUT | E_NO_PAGE | E_BAD_INPUT | E_GOAL_UNCLEAR` — never a stack trace.
+`E_NOT_FOUND | E_TIMEOUT | E_NO_PAGE | E_BAD_INPUT | E_GOAL_UNCLEAR | E_STALE | E_EXPECT | E_BAD_EXPECT | E_MUST_OBSERVE | E_APPROVAL_REQUIRED` — never a stack trace.
+
+## CLI face (same engine, no MCP client needed)
+
+```bash
+effing-use observe --kind snapshot --mode delta --session dev
+effing-use act --action click --target e5 --expect 'url~/dashboard' --session dev
+effing-use extract --kind state --session dev
+```
+
+Set `EFFING_USE_URL` (default `http://localhost:3123/mcp`). Requires a running `bun src/http.ts` server.
+
+## Record → replay
+
+```bash
+# via MCP
+browser_act action=record_start value=my-flow
+# ... do the flow ...
+browser_act action=record_stop
+browser_act action=compile value=my-flow   # -> .browser-use/macros/my-flow.{ts,md}
+browser_act action=replay value=my-flow    # pauses with E_APPROVAL_REQUIRED if irreversible
+browser_act action=replay value=my-flow approve=true
+```
 
 ## Setup
 

package/src/browser/delta.ts

@@ -0,0 +1,136 @@
+import type { Page } from "playwright";
+
+type PageState = {
+  dirty: boolean;
+  baseline: string | null;
+  baselineUrl: string | null;
+};
+
+const pageStates = new Map<string, PageState>();
+
+function key(sessionId: string): string {
+  return sessionId;
+}
+
+export function ensureState(sessionId: string): PageState {
+  let s = pageStates.get(key(sessionId));
+  if (!s) {
+    s = { dirty: true, baseline: null, baselineUrl: null };
+    pageStates.set(key(sessionId), s);
+  }
+  return s;
+}
+
+export function markDirty(sessionId: string): void {
+  ensureState(sessionId).dirty = true;
+}
+
+export function markClean(sessionId: string): void {
+  ensureState(sessionId).dirty = false;
+}
+
+export function setBaseline(
+  sessionId: string,
+  snapshot: string,
+  url: string,
+): void {
+  const s = ensureState(sessionId);
+  s.baseline = snapshot;
+  s.baselineUrl = url;
+  s.dirty = false;
+}
+
+export function getBaseline(sessionId: string): string | null {
+  return ensureState(sessionId).baseline;
+}
+
+export function isDirty(sessionId: string): boolean {
+  return ensureState(sessionId).dirty;
+}
+
+export function clearDelta(sessionId: string): void {
+  pageStates.delete(key(sessionId));
+}
+
+export async function injectDirtyObserver(
+  page: Page,
+  sessionId: string,
+): Promise<void> {
+  try {
+    await page.evaluate((sid) => {
+      const w = window as unknown as {
+        __effDirty?: boolean;
+        __effSid?: string;
+        __effObs?: MutationObserver;
+      };
+      w.__effDirty = false;
+      w.__effSid = sid;
+      if (w.__effObs) w.__effObs.disconnect();
+      const obs = new MutationObserver(() => {
+        w.__effDirty = true;
+      });
+      obs.observe(document.documentElement, {
+        childList: true,
+        subtree: true,
+        attributes: true,
+        characterData: true,
+      });
+      document.addEventListener(
+        "input",
+        () => {
+          w.__effDirty = true;
+        },
+        true,
+      );
+      document.addEventListener(
+        "change",
+        () => {
+          w.__effDirty = true;
+        },
+        true,
+      );
+      w.__effObs = obs;
+    }, sessionId);
+  } catch {
+    // ignore injection failures (e.g. page not ready)
+  }
+}
+
+export async function checkDirty(page: Page): Promise<boolean> {
+  try {
+    const d = await page.evaluate(
+      () => (window as unknown as { __effDirty?: boolean }).__effDirty,
+    );
+    return Boolean(d);
+  } catch {
+    return true;
+  }
+}
+
+export async function clearDirtyFlag(page: Page): Promise<void> {
+  try {
+    await page.evaluate(() => {
+      (window as unknown as { __effDirty?: boolean }).__effDirty = false;
+    });
+  } catch {
+    /* ignore */
+  }
+}
+
+export function computeDelta(
+  baseline: string | null,
+  current: string,
+): { delta: string; unchanged: boolean } {
+  if (!baseline) return { delta: current, unchanged: false };
+  if (baseline === current) return { delta: "", unchanged: true };
+  // Simple line diff: emit only changed lines with [changed] markers
+  const baseLines = baseline.split("\n");
+  const curLines = current.split("\n");
+  const baseSet = new Set(baseLines);
+  const changed = curLines.filter((l) => !baseSet.has(l));
+  const unchangedCount = curLines.length - changed.length;
+  if (changed.length === 0) return { delta: "", unchanged: true };
+  const header = `…${unchangedCount} unchanged lines…`;
+  const delta = [header, ...changed.map((l) => `[changed] ${l}`)].join("\n");
+  return { delta, unchanged: false };
+}