@sightmap/agent-browser 0.9.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Fullstory, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,138 @@
+# @sightmap/agent-browser
+
+Sightmap-aware wrapper around Vercel's [`agent-browser`](https://agent-browser.dev) CLI. Adds four sightmap-aware commands (`snapshot`, `match`, `act`, `network`) that consume your `.sightmap/` corpus; all other commands forward verbatim to `agent-browser` — including the `-p <provider>` flag for hosted Chrome providers.
+
+## Install
+
+    pnpm add -D @sightmap/agent-browser agent-browser
+    agent-browser install   # downloads Chrome on first run
+
+`agent-browser` is a peer dependency. `sightmap-agent-browser` shells out to it for browser primitives, so it has to be on `PATH` (or installed in the same `node_modules`).
+
+## Usage
+
+```sh
+sightmap-agent-browser [global flags] <command> [args...]
+```
+
+### Global flags
+
+| Flag | Default | Meaning |
+| --- | --- | --- |
+| `-s`, `--session NAME` | `default` | Named agent-browser session |
+| `--sightmap-dir DIR` | `.sightmap` | Path to `.sightmap/` directory |
+| `--json` | `false` | Emit JSON instead of human-readable text |
+
+The `.sightmap/` directory is loaded **per invocation** from `--sightmap-dir`, resolved against the current working directory. There is no daemon and no cache — every command re-reads the corpus, so edits to `.sightmap/` are picked up immediately.
+
+## Sightmap-specific commands
+
+Four commands consume `.sightmap/` and produce enriched output. Everything else falls through to `agent-browser`.
+
+### `snapshot` — enriched a11y snapshot
+
+Captures an ARIA snapshot from the current page and annotates it with the matched sightmap view, view memory, and the sightmap components present on the page.
+
+```text
+$ sightmap-agent-browser -s demo snapshot
+View: Example (/**)
+  memory:
+    - IANA-reserved domain used for documentation examples
+Components:
+  - Heading (view-scoped) — 1 match
+
+--- ARIA snapshot ---
+- heading "Example Domain" [level=1, ref=e1]
+- paragraph
+  - link "Learn more" [ref=e2]
+```
+
+Pass `--json` for a structured object including an `ariaSnapshot` field carrying the raw a11y text (parity with `@sightmap/mcp`).
+
+### `match URL` — show the matching sightmap view
+
+Pure query: resolves a URL (or pathname) against the loaded sightmap and prints the matched view, applicable components, and known requests. No browser is launched.
+
+```text
+$ sightmap-agent-browser match /list/abc123
+View: ListDetailScreen (route /list/*)
+  memory: id is opaque; do not parse
+Components:
+  - BottomTabBar (global)
+  - ListCard (view)
+Requests:
+  - GET /api/list/:id → fetchList
+```
+
+### `act NAME` — resolve a component name to a selector
+
+Resolves a sightmap component name to its primary CSS selector (plus any fallbacks). Prints the selector to stdout so it can be piped into another `agent-browser` command. Exits non-zero with a reason for unknown components.
+
+```text
+$ sightmap-agent-browser act SubmitButton
+[data-testid="submit"]
+# fallbacks: button[type="submit"]
+```
+
+Pass `--json` for the full resolution result.
+
+### `network` — annotated network requests
+
+Pulls recent network requests from the current session and tags any that match a `requests:` entry in the sightmap, attaching request memory.
+
+```text
+$ sightmap-agent-browser network
+[GET] https://example.test/api/list/abc → 200  [fetchList]
+    memory: returns 404 for archived lists; treat as "not found"
+[POST] https://example.test/api/track → 204
+```
+
+## Passthrough
+
+Any command not listed above forwards to `agent-browser` verbatim, with `--session NAME` prepended. This makes `sightmap-agent-browser` a drop-in upgrade for `agent-browser`:
+
+    sightmap-agent-browser open https://example.com
+    sightmap-agent-browser click @e3
+    sightmap-agent-browser screenshot page.png
+
+You only opt into the sightmap-aware behavior on the four commands above; everything else still works exactly as `agent-browser` documents it.
+
+## Hosted browser providers
+
+`agent-browser`'s `-p <provider>` flag flows through unchanged. The provider binds to the session on `open`, and all subsequent sightmap-aware commands inherit it:
+
+    sightmap-agent-browser -p browserless -s demo open https://example.com
+    sightmap-agent-browser -s demo snapshot   # runs against the browserless-hosted browser
+
+Sightmap layers above whichever provider you pick (Browserless, Browserbase, Kernel, …) — no extra wrapper code needed, no extra cost.
+
+## Sessions
+
+`agent-browser` uses **named sessions** to keep a browser context alive across invocations. `sightmap-agent-browser` honors the same flag:
+
+```sh
+sightmap-agent-browser -s checkout open https://shop.test
+sightmap-agent-browser -s checkout snapshot
+sightmap-agent-browser -s checkout network
+sightmap-agent-browser -s checkout close
+```
+
+If you don't pass `-s`/`--session`, the session is `"default"`. Run parallel flows by giving them distinct session names.
+
+## `.sightmap/` resolution
+
+Resolved per-invocation, from CWD by default. Override with `--sightmap-dir` when invoking from outside the project root:
+
+```sh
+sightmap-agent-browser --sightmap-dir /path/to/repo/.sightmap snapshot
+```
+
+## Integration tests
+
+Set `SIGHTMAP_AB_INTEGRATION=1` to enable the real-browser smoke test in this package's `test/integration/`. The smoke test runs `open` → `snapshot` → `close` against `https://example.com` and verifies enrichment fires end-to-end.
+
+## See also
+
+- [`@sightmap/playwright`](../playwright) — sister wrapper for `@playwright/cli`. Same four enriched commands, different browser backend.
+- [`@sightmap/mcp`](../mcp) — the MCP-server alternative with the curation tool family.
+- [`@sightmap/sightmap`](../sightmap) — the underlying matcher, lint, and validate library.

package/dist/cli.d.ts

@@ -0,0 +1 @@
+#!/usr/bin/env node

package/dist/cli.js

@@ -0,0 +1,395 @@
+#!/usr/bin/env node
+
+// src/commands/act.ts
+import {
+  loadDirectory,
+  resolveSightmapAct
+} from "@sightmap/sightmap";
+async function runActCommand(opts) {
+  const sightmap = await loadDirectory(opts.sightmapDir);
+  const r = resolveSightmapAct(sightmap, { componentName: opts.componentName });
+  if (r.kind === "error") {
+    process.stderr.write(r.message + "\n");
+    process.exit(1);
+    return;
+  }
+  if (opts.json) {
+    process.stdout.write(JSON.stringify(r, null, 2) + "\n");
+    return;
+  }
+  process.stdout.write(`${r.selector}
+`);
+  if (r.allSelectors.length > 1) {
+    process.stdout.write(`# fallbacks: ${r.allSelectors.slice(1).join(", ")}
+`);
+  }
+}
+
+// src/commands/match.ts
+import { loadDirectory as loadDirectory2, match } from "@sightmap/sightmap";
+async function runMatchCommand(opts) {
+  const sightmap = await loadDirectory2(opts.sightmapDir);
+  const result = match(sightmap, { url: opts.url });
+  if (opts.json) {
+    process.stdout.write(JSON.stringify(result, null, 2) + "\n");
+    return;
+  }
+  const lines = [];
+  if (result.view) {
+    lines.push(`View: ${result.view.name} (${result.view.route})`);
+    for (const m of result.view.memory ?? []) lines.push(`  memory: ${m}`);
+  } else {
+    lines.push(`View: (no match)`);
+  }
+  if (result.components.length > 0) {
+    lines.push(`Components:`);
+    for (const c of result.components) {
+      lines.push(`  - ${c.name} (${c.scope})`);
+    }
+  }
+  if (result.requests.length > 0) {
+    lines.push(`Requests:`);
+    for (const r of result.requests) {
+      lines.push(`  - ${r.method ?? "ANY"} ${r.route} \u2192 ${r.name}`);
+    }
+  }
+  process.stdout.write(lines.join("\n") + "\n");
+}
+
+// src/commands/network.ts
+import {
+  loadDirectory as loadDirectory3,
+  annotateNetworkRequests
+} from "@sightmap/sightmap";
+
+// src/exec.ts
+import { spawn } from "child_process";
+async function execAgentBrowser(args, options = {}) {
+  const binary = options.binary ?? "agent-browser";
+  const timeoutMs = options.timeoutMs ?? 6e4;
+  const wantsStdin = options.stdinInput !== void 0;
+  return await new Promise((resolve, reject) => {
+    const child = spawn(binary, args, {
+      cwd: options.cwd ?? process.cwd(),
+      env: options.env ?? process.env,
+      stdio: [wantsStdin ? "pipe" : "ignore", "pipe", "pipe"]
+    });
+    const stdoutChunks = [];
+    const stderrChunks = [];
+    child.stdout.on("data", (c) => stdoutChunks.push(c));
+    child.stderr.on("data", (c) => stderrChunks.push(c));
+    const killTimer = setTimeout(() => {
+      child.kill("SIGTERM");
+    }, timeoutMs);
+    child.on("error", (err) => {
+      clearTimeout(killTimer);
+      reject(err);
+    });
+    child.on("close", (code) => {
+      clearTimeout(killTimer);
+      resolve({
+        exitCode: code ?? -1,
+        stdout: Buffer.concat(stdoutChunks).toString("utf8"),
+        stderr: Buffer.concat(stderrChunks).toString("utf8")
+      });
+    });
+    if (wantsStdin && child.stdin) {
+      child.stdin.write(options.stdinInput);
+      child.stdin.end();
+    }
+  });
+}
+async function streamAgentBrowser(args, options = {}) {
+  const binary = options.binary ?? "agent-browser";
+  return await new Promise((resolve, reject) => {
+    const child = spawn(binary, args, {
+      cwd: options.cwd ?? process.cwd(),
+      env: options.env ?? process.env,
+      stdio: "inherit"
+    });
+    child.on("error", reject);
+    child.on("close", (code) => {
+      resolve({ exitCode: code ?? -1 });
+    });
+  });
+}
+
+// src/parse-network.ts
+function parseAgentBrowserNetwork(text) {
+  const env = JSON.parse(text);
+  if (env.success !== true) {
+    throw new Error(
+      `agent-browser network --json: ${env.error ?? "unknown error"}`
+    );
+  }
+  const data = env.data ?? [];
+  const entries = Array.isArray(data) ? data : Array.isArray(data.requests) ? data.requests : [];
+  return entries.map((e) => ({
+    method: e.method ?? "",
+    url: e.url ?? "",
+    status: typeof e.status === "number" ? e.status : 0,
+    statusText: e.statusText ?? ""
+  }));
+}
+
+// src/commands/network.ts
+async function runNetworkCommand(opts) {
+  const sightmap = await loadDirectory3(opts.sightmapDir);
+  const result = await execAgentBrowser([
+    "--session",
+    opts.sessionName,
+    "network",
+    "requests",
+    "--json"
+  ]);
+  if (result.exitCode !== 0) {
+    process.stderr.write(result.stderr);
+    process.exit(result.exitCode);
+  }
+  const parsed = parseAgentBrowserNetwork(result.stdout);
+  const annotated = annotateNetworkRequests(sightmap, parsed);
+  if (opts.json) {
+    process.stdout.write(JSON.stringify(annotated, null, 2) + "\n");
+    return;
+  }
+  for (const r of annotated) {
+    const tag = r.sightmapName ? `  [${r.sightmapName}]` : "";
+    process.stdout.write(`[${r.method}] ${r.url} \u2192 ${r.status}${tag}
+`);
+    if (r.sightmapMemory) {
+      for (const m of r.sightmapMemory) {
+        process.stdout.write(`    memory: ${m}
+`);
+      }
+    }
+  }
+}
+
+// src/commands/passthrough.ts
+async function runPassthroughCommand(opts) {
+  const { exitCode } = await streamAgentBrowser([
+    "--session",
+    opts.sessionName,
+    ...opts.rest
+  ]);
+  if (exitCode !== 0) process.exit(exitCode);
+}
+
+// src/commands/snapshot.ts
+import {
+  enrichSnapshot,
+  loadDirectory as loadDirectory4,
+  match as sightmapMatch
+} from "@sightmap/sightmap";
+
+// src/parse-snapshot.ts
+function parseAgentBrowserSnapshot(text) {
+  const env = JSON.parse(text);
+  if (env.success !== true) {
+    throw new Error(
+      `agent-browser snapshot --json: ${env.error ?? "unknown error"}`
+    );
+  }
+  const data = env.data ?? {};
+  return {
+    url: data.origin ?? "",
+    ariaText: data.snapshot ?? "",
+    refs: data.refs ?? {}
+  };
+}
+
+// src/in-page-runner.ts
+import {
+  buildInPageEvalFunction
+} from "@sightmap/sightmap";
+function parseAgentBrowserEvalEnvelope(stdout) {
+  const env = JSON.parse(stdout);
+  if (env.success !== true) {
+    throw new Error(
+      `agent-browser eval --json: ${env.error ?? "unknown error"}`
+    );
+  }
+  const matches = env.data?.result?.matches;
+  return Array.isArray(matches) ? matches : [];
+}
+async function runInPageMatcher(components, sessionName, options = {}) {
+  if (components.length === 0) return [];
+  const js = buildInPageEvalFunction(components);
+  const iife = `(${js})()`;
+  const result = await execAgentBrowser(
+    ["--session", sessionName, "eval", "--stdin", "--json"],
+    { ...options, stdinInput: iife }
+  );
+  if (result.exitCode !== 0) {
+    throw new Error(`agent-browser eval failed: ${result.stderr}`);
+  }
+  return parseAgentBrowserEvalEnvelope(result.stdout);
+}
+
+// src/commands/snapshot.ts
+async function runSnapshotCommand(opts) {
+  const sightmap = await loadDirectory4(opts.sightmapDir);
+  const snapResult = await execAgentBrowser([
+    "--session",
+    opts.sessionName,
+    "snapshot",
+    "--json"
+  ]);
+  if (snapResult.exitCode !== 0) {
+    process.stderr.write(snapResult.stderr);
+    process.exit(snapResult.exitCode);
+  }
+  const parsed = parseAgentBrowserSnapshot(snapResult.stdout);
+  const matchResult = sightmapMatch(sightmap, { url: parsed.url });
+  const allComponents = matchResult.components.map((c) => ({
+    name: c.name,
+    selector: c.selector
+  }));
+  const inPage = await runInPageMatcher(allComponents, opts.sessionName);
+  const enriched = enrichSnapshot({
+    sightmap,
+    currentUrl: parsed.url,
+    inPageMatches: inPage
+  });
+  if (opts.json) {
+    process.stdout.write(
+      JSON.stringify(
+        { ...enriched, ariaSnapshot: parsed.ariaText },
+        null,
+        2
+      ) + "\n"
+    );
+  } else {
+    process.stdout.write(formatHuman(enriched, parsed.ariaText));
+  }
+}
+function formatHuman(enriched, ariaText) {
+  const lines = [];
+  if (enriched.view) {
+    lines.push(`View: ${enriched.view.name} (${enriched.view.route})`);
+    if (enriched.view.memory.length > 0) {
+      lines.push(`  memory:`);
+      for (const m of enriched.view.memory) lines.push(`    - ${m}`);
+    }
+  } else {
+    lines.push(`View: (no match)`);
+  }
+  if (enriched.components.length > 0) {
+    lines.push(`Components:`);
+    for (const c of enriched.components) {
+      lines.push(
+        `  - ${c.name} (${c.scope}) \u2014 ${c.matchCount} match${c.matchCount === 1 ? "" : "es"}`
+      );
+      for (const m of c.memory) lines.push(`      memory: ${m}`);
+    }
+  }
+  lines.push("");
+  lines.push("--- ARIA snapshot ---");
+  lines.push(ariaText);
+  return lines.join("\n");
+}
+
+// src/cli.ts
+function requireValue(flag, next) {
+  if (next === void 0 || next.startsWith("-")) {
+    console.error(
+      `${flag}: expected a value, got "${next ?? "(end of args)"}"`
+    );
+    process.exit(2);
+  }
+  return next;
+}
+function parseArgs(argv) {
+  let session = "default";
+  let sightmapDir = ".sightmap";
+  let json = false;
+  const positional = [];
+  for (let i = 0; i < argv.length; i++) {
+    const a = argv[i];
+    if (a === "-s" || a === "--session") {
+      session = requireValue(a, argv[++i]);
+    } else if (a === "--sightmap-dir") {
+      sightmapDir = requireValue(a, argv[++i]);
+    } else if (a === "--json") {
+      json = true;
+    } else {
+      positional.push(a);
+    }
+  }
+  return {
+    command: positional[0] ?? "",
+    session,
+    sightmapDir,
+    json,
+    rest: positional.slice(1)
+  };
+}
+function printHelp() {
+  console.log(
+    `sightmap-agent-browser \u2014 sightmap-aware wrapper around Vercel's agent-browser CLI
+
+Usage:
+  sightmap-agent-browser [global flags] <command> [args...]
+
+Global flags:
+  -s, --session NAME      Named agent-browser session (default: "default")
+  --sightmap-dir DIR      Path to .sightmap/ directory (default: ".sightmap")
+  --json                  Emit JSON instead of human-readable text
+
+Sightmap-specific commands:
+  snapshot                Enriched a11y snapshot with sightmap component names
+  match URL               Show the matching sightmap view for URL
+  act NAME                Resolve a sightmap component name to a selector
+  network                 Network requests annotated with sightmap memory
+
+All other commands are forwarded to agent-browser verbatim, including
+the -p <provider> flag for hosted browser providers.
+`
+  );
+}
+async function main() {
+  const argv = process.argv.slice(2);
+  if (argv.length === 0 || argv[0] === "--help" || argv[0] === "-h") {
+    printHelp();
+    return;
+  }
+  const { command, session, sightmapDir, json, rest } = parseArgs(argv);
+  switch (command) {
+    case "snapshot":
+      await runSnapshotCommand({ sessionName: session, sightmapDir, json });
+      return;
+    case "match": {
+      const url = rest[0];
+      if (url === void 0) {
+        console.error("sightmap-agent-browser match: URL required");
+        process.exit(2);
+      }
+      await runMatchCommand({ url, sightmapDir, json });
+      return;
+    }
+    case "act": {
+      const name = rest[0];
+      if (name === void 0) {
+        console.error("sightmap-agent-browser act: component name required");
+        process.exit(2);
+      }
+      await runActCommand({ componentName: name, sightmapDir, json });
+      return;
+    }
+    case "network":
+      await runNetworkCommand({ sessionName: session, sightmapDir, json });
+      return;
+    default: {
+      const passArgs = json ? ["--json", command, ...rest] : [command, ...rest];
+      await runPassthroughCommand({
+        sessionName: session,
+        rest: passArgs.filter((s) => s.length > 0)
+      });
+      return;
+    }
+  }
+}
+main().catch((err) => {
+  console.error(err instanceof Error ? err.message : String(err));
+  process.exit(1);
+});

package/package.json

@@ -0,0 +1,55 @@
+{
+  "name": "@sightmap/agent-browser",
+  "version": "0.9.0",
+  "description": "Sightmap-aware wrapper around Vercel's agent-browser CLI. Adds sightmap-* commands and enriches snapshot output with component names from .sightmap/.",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/sightmap/sightmap-js.git",
+    "directory": "packages/agent-browser"
+  },
+  "homepage": "https://sightmap.org",
+  "keywords": [
+    "sightmap",
+    "agent-browser",
+    "cli",
+    "agent",
+    "browser"
+  ],
+  "type": "module",
+  "bin": {
+    "sightmap-agent-browser": "./dist/cli.js"
+  },
+  "files": [
+    "dist",
+    "README.md",
+    "LICENSE"
+  ],
+  "publishConfig": {
+    "access": "public"
+  },
+  "devDependencies": {
+    "agent-browser": "^0.27.0",
+    "@types/node": "^20.19.39",
+    "tsup": "^8.5.1",
+    "typescript": "~5.7.3",
+    "vitest": "^3.0.0"
+  },
+  "dependencies": {
+    "@sightmap/sightmap": "^0.9.0"
+  },
+  "peerDependencies": {
+    "agent-browser": "^0.27.0"
+  },
+  "peerDependenciesMeta": {
+    "agent-browser": {
+      "optional": false
+    }
+  },
+  "scripts": {
+    "build": "tsup",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "typecheck": "tsc --noEmit"
+  }
+}