feed-the-machine 1.0.0

package/ftm-browse/SKILL.md

@@ -0,0 +1,415 @@
+---
+name: ftm-browse
+description: Headless browser daemon for visual verification and web interaction. Gives agents the ability to navigate, screenshot, click, fill forms, and inspect ARIA trees via CLI commands. Use when user says "browse", "screenshot", "visual", "look at the app", "open browser", "check the page", "navigate to", "take a screenshot", "visual verification".
+---
+
+## Events
+
+### Emits
+- `task_completed` — when a visual verification or interaction workflow finishes successfully
+
+### Listens To
+(none — ftm-browse is invoked on demand and does not respond to events)
+
+# ftm-browse
+
+ftm-browse is a persistent headless Chromium daemon controlled via a CLI binary at `~/.claude/skills/ftm-browse/bin/ftm-browse`. Each CLI invocation communicates with the daemon over a local HTTP server (bearer-auth, random port), so the browser stays alive across commands without the per-invocation startup penalty. The daemon auto-starts on first use and shuts itself down after 30 minutes of idle. This CLI-to-HTTP model is 4x more token-efficient than driving Playwright MCP directly, because tool calls remain terse and outputs are structured JSON rather than raw browser protocol noise.
+
+---
+
+## Setup
+
+**First run — install the browser engine:**
+
+```bash
+npx playwright install chromium
+```
+
+**Define the alias in any shell session before use:**
+
+```bash
+PB="$HOME/.claude/skills/ftm-browse/bin/ftm-browse"
+```
+
+**Verify the installation:**
+
+```bash
+$PB goto https://example.com && $PB screenshot
+```
+
+The first `goto` command will start the daemon (up to 10 seconds for cold start). All subsequent commands respond in ~100ms because the browser process stays alive.
+
+---
+
+## Command Reference
+
+### WRITE commands — state-mutating
+
+These commands change browser state. Never retry blindly; check the returned `success` field.
+
+**`goto <url>`** — Navigate to a URL. Waits for `domcontentloaded`.
+
+```bash
+$PB goto https://example.com
+$PB goto http://localhost:3000/dashboard
+```
+
+Returns: `{ success, data: { url, title, status } }`
+
+**`click <@ref>`** — Click an interactive element by its `@e` ref. Performs a 5ms staleness check before clicking; fails immediately if the ref is stale rather than waiting.
+
+```bash
+$PB click @e3
+$PB click @e12
+```
+
+Returns: `{ success, data: { url, title } }` — includes the URL after any resulting navigation.
+
+**`fill <@ref> <value>`** — Fill a text input, textarea, or other fillable element. Values with spaces do not need quoting — remaining CLI args are joined.
+
+```bash
+$PB fill @e2 hello world
+$PB fill @e5 user@example.com
+```
+
+Returns: `{ success, data: { ref, value } }`
+
+**`press <key>`** — Send a keyboard key to the active page. Accepts any Playwright key name.
+
+```bash
+$PB press Enter
+$PB press Tab
+$PB press Escape
+$PB press ArrowDown
+```
+
+Returns: `{ success, data: { key, url } }`
+
+---
+
+### READ commands — safe to retry
+
+These commands do not change browser state. Safe to call multiple times.
+
+**`text`** — Get visible page text via `document.body.innerText`.
+
+```bash
+$PB text
+```
+
+Returns: `{ success, data: { text } }`
+
+**`html`** — Get full page HTML via `page.content()`.
+
+```bash
+$PB html
+```
+
+Returns: `{ success, data: { html } }`
+
+**`eval <js-expression>`** — Execute arbitrary JavaScript in the page context. Returns the result as a JSON-serializable value. DOM elements are returned as `{ type: "element", tagName, id, text }`. Functions are returned as `{ type: "function", name }`. Errors inside the expression are caught and returned as `{ error: "..." }` rather than crashing the daemon.
+
+```bash
+$PB eval document.title
+$PB eval "document.querySelector('input[name=email]').value"
+$PB eval "Array.from(document.querySelectorAll('td')).map(td => td.textContent)"
+$PB eval window.location.href
+$PB eval "document.cookie"
+```
+
+Returns: `{ success, data: { result } }` — `result` is whatever the expression evaluated to.
+
+Use cases:
+- Reading hidden or programmatically-set form field values not visible in the ARIA tree
+- Extracting data from dynamic tables or rendered lists
+- Checking feature toggle states or global JS variables (`window.featureFlags`)
+- Inspecting `localStorage` or `sessionStorage` values
+- Querying computed DOM state not exposed via accessibility roles
+
+---
+
+### META commands
+
+**`snapshot`** — Full ARIA accessibility tree of the current page. Includes both interactive and structural elements (headings, nav, main, etc.).
+
+```bash
+$PB snapshot
+```
+
+Returns: `{ success, data: { url, title, interactive_only: false, tree, refs, aria_text? } }`
+
+**`snapshot -i`** — Interactive elements only, each labeled with an `@e1`, `@e2`... ref. Use this before clicking or filling — never guess a ref.
+
+```bash
+$PB snapshot -i
+```
+
+Returns: same shape as `snapshot` with `interactive_only: true`; the `refs` map contains the locator entries for each `@eN`.
+
+**`screenshot`** — Capture a viewport screenshot (1280x800). Saves to `~/.ftm-browse/screenshots/screenshot-<timestamp>.png` by default and returns the path.
+
+```bash
+$PB screenshot
+$PB screenshot --path /tmp/before.png
+$PB screenshot --path /tmp/after.png
+```
+
+Returns: `{ success, data: { path, url, title } }`
+
+**`tabs`** — List all open browser tabs.
+
+```bash
+$PB tabs
+```
+
+Returns: `{ success, data: { tabs: [{ index, url, title, active }] } }`
+
+**`chain '<json-array>'`** — Execute multiple commands in sequence in a single CLI invocation. The chain stops at the first failure. Use this to reduce round-trips for multi-step operations.
+
+```bash
+$PB chain '[
+  {"command":"goto","args":{"url":"https://example.com"}},
+  {"command":"snapshot","args":{"interactive_only":true}},
+  {"command":"screenshot","args":{}}
+]'
+```
+
+Returns: `{ success, data: { results: [{ command, result }] } }`. On failure: adds `failed_at` field.
+
+**`health`** — Check that the daemon is alive and responding.
+
+```bash
+$PB health
+```
+
+Returns: `{ status: "ok", pid }` (wrapped in standard result envelope from the daemon's health handler — note this endpoint bypasses `executeCommand` and returns directly).
+
+**`stop`** (alias: `shutdown`) — Send SIGTERM to the daemon. The daemon cleans up its state file and exits.
+
+```bash
+$PB stop
+```
+
+---
+
+## The @e Ref System
+
+Refs are short handles (`@e1`, `@e2`, ...) that identify interactive elements. They are assigned fresh on each `snapshot` call and map to stable Playwright locator strategies (by label, by role+name, by placeholder, by name attribute, or by nth-position CSS fallback).
+
+**Getting refs:**
+
+```bash
+$PB snapshot -i
+# Output includes tree nodes like:
+# { "ref": "@e3", "role": "button", "name": "Submit", "interactive": true }
+# { "ref": "@e5", "role": "textbox", "name": "Email", "interactive": true }
+```
+
+**Using refs:**
+
+```bash
+$PB fill @e5 user@example.com
+$PB click @e3
+```
+
+**Staleness rule:** After any navigation event — whether from `goto`, a `click` that follows a link, or `press Enter` submitting a form — the current ref map is invalidated. The daemon detects stale refs in ~5ms and returns an error asking you to re-snapshot. Always re-run `snapshot -i` after navigation before using refs again.
+
+**Typical interaction workflow:**
+
+```bash
+# 1. Navigate to the page
+$PB goto http://localhost:3000/login
+
+# 2. Get interactive refs
+$PB snapshot -i
+
+# 3. Identify target elements from the output, then interact
+$PB fill @e2 admin@example.com
+$PB fill @e3 password123
+$PB click @e4          # "Sign in" button
+
+# 4. Page navigated — refs are stale; re-snapshot
+$PB snapshot -i
+
+# 5. Continue on the new page
+$PB screenshot
+```
+
+---
+
+## Common Workflows
+
+### Visual smoke test
+
+```bash
+PB="$HOME/.claude/skills/ftm-browse/bin/ftm-browse"
+$PB goto http://localhost:3000
+$PB screenshot --path /tmp/smoke.png
+# Read /tmp/smoke.png to verify layout
+```
+
+### Form filling
+
+```bash
+$PB goto http://localhost:3000/signup
+$PB snapshot -i
+# Identify: @e1=name input, @e2=email input, @e3=password, @e4=submit button
+$PB fill @e1 Jane Doe
+$PB fill @e2 jane@example.com
+$PB fill @e3 s3cret!
+$PB click @e4
+$PB screenshot --path /tmp/after-signup.png
+```
+
+### Navigation verification
+
+```bash
+$PB goto http://localhost:3000
+$PB snapshot -i
+# Find nav links
+$PB click @e7          # "Dashboard" link
+$PB text               # Verify content changed
+$PB screenshot
+```
+
+### Before/after comparison
+
+```bash
+$PB goto http://localhost:3000/widget
+$PB screenshot --path /tmp/before.png
+# ... make changes in code ...
+$PB goto http://localhost:3000/widget   # reload after change
+$PB screenshot --path /tmp/after.png
+# Compare /tmp/before.png and /tmp/after.png visually
+```
+
+### Multi-step with chain (fewer round-trips)
+
+```bash
+$PB chain '[
+  {"command":"goto","args":{"url":"http://localhost:3000/login"}},
+  {"command":"fill","args":{"ref":"@e2","value":"admin@example.com"}},
+  {"command":"fill","args":{"ref":"@e3","value":"password"}},
+  {"command":"click","args":{"ref":"@e4"}},
+  {"command":"screenshot","args":{}}
+]'
+```
+
+Note: When using `chain` with refs, you must have called `snapshot -i` first in a separate command to populate the ref map. Refs set by a `snapshot` inside the same chain are available to subsequent steps in that chain.
+
+---
+
+## Integration with Other FTM Skills
+
+**ftm-debug** — Use ftm-browse to visually verify bug fixes. Take a screenshot before applying a fix, apply the fix, reload, screenshot again. Compare before/after to confirm the fix is visible. Also use `snapshot` to inspect DOM state when debugging rendering issues — the ARIA tree reveals whether components have mounted and populated correctly.
+
+**ftm-audit** — Use ftm-browse to verify runtime wiring. Navigate to each route the audit is checking, call `snapshot` to confirm the component appears in the ARIA tree with the correct role and name, and screenshot for documentation. This catches hydration failures, missing route registrations, and components that render blank.
+
+**ftm-executor** — After completing a task that touches frontend code, use ftm-browse as the post-task smoke test harness. If the project has a dev server running, `goto` the affected route, take a screenshot, and verify the page renders without errors. Include the screenshot path in the task completion report.
+
+---
+
+## Supervised Execution Mode
+
+When ftm-browse is executing browser steps within an approved plan (dispatched by ftm-executor or ftm-mind), activate supervised mode. This mode adds verification guardrails after every navigation action.
+
+### Activation
+
+Supervised mode activates automatically when:
+- The browse command is part of a plan step (context includes plan step reference)
+- The caller provides expected page state (title pattern, URL pattern, or element selector)
+
+### Post-Navigation Verification
+
+After every `goto`, `click` that triggers navigation, or `fill` + `submit` sequence:
+
+1. **Wait for page load** — wait for `networkidle` or 5-second timeout, whichever comes first
+2. **Check URL** — if expected URL pattern was provided, verify current URL matches
+3. **Check title** — if expected title pattern was provided, verify page title matches
+4. **Check for error indicators** — scan for common error patterns:
+   - HTTP error codes in title (403, 404, 500, 502, 503)
+   - Error modals or alert dialogs
+   - "Access Denied", "Unauthorized", "Session Expired" text in page
+5. **Check for auth redirects** — if current URL contains `/login`, `/signin`, `/sso`, `/oauth`, or `/saml` when it wasn't the intended destination, flag as auth redirect
+
+### On Unexpected State
+
+When verification detects a mismatch or error:
+
+1. **Stop execution immediately** — do not proceed to the next browser step
+2. **Take a screenshot** — capture the current page state
+3. **Present the situation** to the user:
+
+```
+⚠ Unexpected browser state during plan step [N]:
+
+Expected: [expected URL/title/state]
+Actual: [current URL/title]
+Issue: [description — wrong page / error page / auth redirect / modal detected]
+
+Screenshot: [path to screenshot]
+
+Options:
+  1. retry  — navigate again
+  2. skip   — skip this browser step, continue plan
+  3. abort  — stop plan execution entirely
+  4. manual — open interactive browser for manual intervention
+```
+
+4. **Wait for user choice** — do not proceed without explicit selection
+
+### Auth Redirect Detection
+
+Authentication redirects are especially dangerous because silently following them can:
+- Leak credentials to unexpected domains
+- Complete OAuth flows the user didn't intend
+- Grant permissions the user didn't approve
+
+When an auth redirect is detected:
+- NEVER automatically follow it
+- Flag it prominently: "Auth redirect detected — redirected to [domain]"
+- The user must explicitly choose to proceed
+
+### Audit Trail
+
+Every browser step within a plan produces:
+- **Before screenshot** — taken just before the action
+- **After screenshot** — taken after the action completes (or after error)
+- **Timing** — how long the action took
+- **Verification result** — PASS or FAIL with details
+
+Screenshots saved to a temp directory, paths included in the step report.
+
+### Non-Plan Usage
+
+When ftm-browse is used directly (not within a plan), supervised mode is OFF by default. The user gets the raw browse experience. They can enable it manually with `--supervised` flag.
+
+---
+
+## Error Handling
+
+| Symptom | Cause | Fix |
+|---|---|---|
+| First command hangs up to 10s | Daemon cold start | Normal — wait for it |
+| `Ref @eN not found. The page may have changed` | Stale ref after navigation | Re-run `snapshot -i` |
+| `Ref @eN no longer exists on the page` | Element removed from DOM | Re-run `snapshot -i` |
+| `Timeout` on goto | Page slow to load or wrong URL | Check URL, verify server is running |
+| `Browser not installed` or Chromium launch error | Playwright Chromium missing | Run `npx playwright install chromium` |
+| `Daemon failed to start within 10 seconds` | Bun or binary issue | Check `~/.ftm-browse/` for logs; verify binary is executable |
+| Connection refused | Daemon died (idle timeout or crash) | Next command will auto-restart it |
+| `commands must be an array` | Bad JSON passed to chain | Validate JSON before passing to chain |
+| `Evaluation failed: ...` | Playwright could not serialize or run the expression | Check for syntax errors; wrap complex expressions in quotes |
+
+---
+
+## Tips
+
+- Always run `snapshot -i` before `click` or `fill` — never guess or hardcode a ref number.
+- Use `chain` for multi-step flows to reduce round-trip overhead; each step result is available in the returned array.
+- Screenshots are cheap — take them liberally at key points (before interaction, after submit, after navigation) as a natural audit trail.
+- The daemon persists across all commands in a session. Cold start only happens once per 30-minute idle window.
+- `$PB text` is the fastest way to assert page content without parsing HTML.
+- `$PB html` is useful when you need to inspect the raw DOM, check for hidden elements, or verify server-rendered markup.
+- The daemon uses a 1280x800 headless Chromium viewport with a standard Mac Chrome user-agent, so most sites render predictably.
+- To stop the daemon explicitly: `$PB stop`. It will auto-restart on next use.
+- `$PB eval` is the escape hatch for anything the ARIA tree doesn't expose — hidden inputs, JS globals, localStorage, computed values.

package/ftm-browse/daemon/browser-manager.ts

@@ -0,0 +1,206 @@
+import * as fs from "fs";
+import * as path from "path";
+import { chromium, type Browser, type BrowserContext, type Page } from "playwright";
+
+const IDLE_TIMEOUT_MS = 30 * 60 * 1000; // 30 minutes
+const DEFAULT_USER_DATA_DIR = path.join(process.env.HOME || "~", ".ftm-browse", "user-data");
+
+class BrowserManager {
+  private browser: Browser | null = null;
+  private context: BrowserContext | null = null;
+  private page: Page | null = null;
+  private idleTimer: ReturnType<typeof setTimeout> | null = null;
+  private shutdownCallback: (() => void) | null = null;
+  private initPromise: Promise<void> | null = null;
+  private userDataDir: string | null = null;
+
+  setUserDataDir(dir: string | null): void {
+    this.userDataDir = dir;
+  }
+
+  setShutdownCallback(cb: () => void): void {
+    this.shutdownCallback = cb;
+  }
+
+  resetIdleTimer(): void {
+    if (this.idleTimer) {
+      clearTimeout(this.idleTimer);
+    }
+    this.idleTimer = setTimeout(() => {
+      console.log("[browser-manager] 30min idle timeout reached, shutting down");
+      void this.shutdown();
+    }, IDLE_TIMEOUT_MS);
+  }
+
+  async initialize(): Promise<void> {
+    // Short-circuit if already running (handles both ephemeral and persistent context modes)
+    if (this.isRunning()) return;
+
+    // Deduplicate concurrent init calls
+    if (this.initPromise) {
+      return this.initPromise;
+    }
+
+    this.initPromise = this._doInit().finally(() => {
+      this.initPromise = null;
+    });
+    return this.initPromise;
+  }
+
+  private async _doInit(): Promise<void> {
+    const launchArgs = [
+      "--no-sandbox",
+      "--disable-setuid-sandbox",
+      "--disable-dev-shm-usage",
+      "--disable-gpu",
+      "--disable-background-networking",
+      "--disable-default-apps",
+      "--disable-extensions",
+      "--disable-sync",
+      "--disable-translate",
+      "--metrics-recording-only",
+      "--mute-audio",
+      "--no-first-run",
+      "--safebrowsing-disable-auto-update",
+    ];
+
+    const contextOptions = {
+      viewport: { width: 1280, height: 800 } as { width: number; height: number } | null,
+      userAgent:
+        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
+    };
+
+    if (this.userDataDir) {
+      // Persistent context — cookies and session data survive daemon restarts
+      const resolvedDir = this.userDataDir === "default" ? DEFAULT_USER_DATA_DIR : this.userDataDir;
+      console.log(`[browser-manager] Launching Chromium with persistent context at: ${resolvedDir}`);
+
+      if (!fs.existsSync(resolvedDir)) {
+        fs.mkdirSync(resolvedDir, { recursive: true, mode: 0o700 });
+      }
+
+      // launchPersistentContext returns a BrowserContext directly (no separate Browser)
+      this.context = await chromium.launchPersistentContext(resolvedDir, {
+        headless: true,
+        args: launchArgs,
+        ...contextOptions,
+      });
+
+      // Reuse existing page if available, otherwise open a new one
+      const existingPages = this.context.pages();
+      this.page = existingPages.length > 0 ? existingPages[0] : await this.context.newPage();
+    } else {
+      // Ephemeral context — no persistence (default, safe)
+      console.log("[browser-manager] Launching Chromium (ephemeral)...");
+
+      this.browser = await chromium.launch({
+        headless: true,
+        args: launchArgs,
+      });
+
+      this.context = await this.browser.newContext(contextOptions);
+      this.page = await this.context.newPage();
+    }
+
+    // Start idle timer
+    this.resetIdleTimer();
+
+    console.log("[browser-manager] Browser ready");
+  }
+
+  async getPage(): Promise<Page> {
+    await this.initialize();
+    this.resetIdleTimer();
+
+    if (!this.page || this.page.isClosed()) {
+      console.log("[browser-manager] Page was closed, creating new page");
+      this.page = await this.context!.newPage();
+    }
+
+    return this.page;
+  }
+
+  async getOrCreatePage(tabIndex?: number): Promise<Page> {
+    await this.initialize();
+    this.resetIdleTimer();
+
+    const pages = this.context!.pages();
+
+    if (tabIndex !== undefined && tabIndex < pages.length) {
+      this.page = pages[tabIndex];
+      return this.page;
+    }
+
+    if (!this.page || this.page.isClosed()) {
+      this.page = await this.context!.newPage();
+    }
+
+    return this.page;
+  }
+
+  async getAllPages(): Promise<Page[]> {
+    if (!this.context) return [];
+    return this.context.pages();
+  }
+
+  async setActivePage(index: number): Promise<Page | null> {
+    if (!this.context) return null;
+    const pages = this.context.pages();
+    if (index < 0 || index >= pages.length) return null;
+    this.page = pages[index];
+    this.resetIdleTimer();
+    return this.page;
+  }
+
+  async shutdown(): Promise<void> {
+    if (this.idleTimer) {
+      clearTimeout(this.idleTimer);
+      this.idleTimer = null;
+    }
+
+    try {
+      if (this.page && !this.page.isClosed()) {
+        await this.page.close();
+      }
+    } catch {
+      // Ignore close errors
+    }
+
+    try {
+      if (this.context) {
+        await this.context.close();
+      }
+    } catch {
+      // Ignore close errors
+    }
+
+    try {
+      if (this.browser) {
+        await this.browser.close();
+      }
+    } catch {
+      // Ignore close errors
+    }
+
+    this.page = null;
+    this.context = null;
+    this.browser = null;
+
+    console.log("[browser-manager] Browser shut down");
+
+    if (this.shutdownCallback) {
+      this.shutdownCallback();
+    }
+  }
+
+  isRunning(): boolean {
+    // Persistent context mode: browser is null, check context instead
+    if (this.userDataDir) {
+      return this.context !== null;
+    }
+    return this.browser !== null && this.browser.isConnected();
+  }
+}
+
+// Singleton instance
+export const browserManager = new BrowserManager();

package/ftm-browse/daemon/bun.lock

@@ -0,0 +1,30 @@
+{
+  "lockfileVersion": 1,
+  "configVersion": 1,
+  "workspaces": {
+    "": {
+      "name": "ftm-browse",
+      "dependencies": {
+        "playwright": "^1.49.0",
+      },
+      "devDependencies": {
+        "@types/bun": "latest",
+      },
+    },
+  },
+  "packages": {
+    "@types/bun": ["@types/bun@1.3.10", "", { "dependencies": { "bun-types": "1.3.10" } }, "sha512-0+rlrUrOrTSskibryHbvQkDOWRJwJZqZlxrUs1u4oOoTln8+WIXBPmAuCF35SWB2z4Zl3E84Nl/D0P7803nigQ=="],
+
+    "@types/node": ["@types/node@25.5.0", "", { "dependencies": { "undici-types": "~7.18.0" } }, "sha512-jp2P3tQMSxWugkCUKLRPVUpGaL5MVFwF8RDuSRztfwgN1wmqJeMSbKlnEtQqU8UrhTmzEmZdu2I6v2dpp7XIxw=="],
+
+    "bun-types": ["bun-types@1.3.10", "", { "dependencies": { "@types/node": "*" } }, "sha512-tcpfCCl6XWo6nCVnpcVrxQ+9AYN1iqMIzgrSKYMB/fjLtV2eyAVEg7AxQJuCq/26R6HpKWykQXuSOq/21RYcbg=="],
+
+    "fsevents": ["fsevents@2.3.2", "", { "os": "darwin" }, "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA=="],
+
+    "playwright": ["playwright@1.58.2", "", { "dependencies": { "playwright-core": "1.58.2" }, "optionalDependencies": { "fsevents": "2.3.2" }, "bin": { "playwright": "cli.js" } }, "sha512-vA30H8Nvkq/cPBnNw4Q8TWz1EJyqgpuinBcHET0YVJVFldr8JDNiU9LaWAE1KqSkRYazuaBhTpB5ZzShOezQ6A=="],
+
+    "playwright-core": ["playwright-core@1.58.2", "", { "bin": { "playwright-core": "cli.js" } }, "sha512-yZkEtftgwS8CsfYo7nm0KE8jsvm6i/PTgVtB8DL726wNf6H2IMsDuxCpJj59KDaxCtSnrWan2AeDqM7JBaultg=="],
+
+    "undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="],
+  }
+}