unbrowse 2.11.0 → 2.12.1

package/README.md

@@ -2,14 +2,43 @@
 
 This package installs the `unbrowse` CLI.
 
-Turn any website into a reusable API interface for agents. Unbrowse captures network traffic, reverse-engineers the real endpoints underneath the UI, and stores what it learns in a shared marketplace so the next agent can reuse it instantly.
+Unbrowse is a local Model Context Protocol (MCP) server and CLI that turns any website into a reusable API interface for agents. It captures network traffic, reverse-engineers the real endpoints underneath the UI, and stores what it learns in a shared marketplace so the next agent can reuse it instantly.
 
 One agent learns a site once. Every later agent gets the fast path.
 
-Unbrowse is a drop-in browser for agents: same browser-shaped job in the stack, but with route learning, reuse, and browser fallback built in.
+Unbrowse is a drop-in replacement for OpenClaw / `agent-browser` browser flows for agents: on the API-native path it is typically ~30x faster, ~90% cheaper, and turns repeated browser work into reusable route assets.
 
 > Security note: capture and execution stay local by default. Credentials stay on your machine. Learned API contracts are published to the shared marketplace only after capture. See [SKILL.md](./SKILL.md) for the full agent-facing API reference and tool-policy guidance.
 
+## MCP server
+
+Unbrowse implements the Model Context Protocol over stdio. `unbrowse mcp` is the MCP server entrypoint.
+
+- Protocol: JSON-RPC 2.0 MCP over stdio
+- Handshake: `initialize`, `notifications/initialized`, `ping`
+- Capability surface today: `tools/list` and `tools/call`
+- Current MCP shape: tools only. No `resources/*` or `prompts/*` methods yet.
+- Runtime model: the MCP server fronts the local Unbrowse runtime on `http://localhost:6969`; hosts talk standard MCP, and Unbrowse uses the local HTTP runtime behind the scenes.
+
+Core MCP tools:
+
+- Discovery: `unbrowse_health`, `unbrowse_search`, `unbrowse_resolve`, `unbrowse_execute`, `unbrowse_feedback`
+- Auth/cache: `unbrowse_login`, `unbrowse_skills`, `unbrowse_skill`, `unbrowse_sessions`
+- Browser capture: `unbrowse_go`, `unbrowse_snap`, `unbrowse_click`, `unbrowse_fill`, `unbrowse_type`, `unbrowse_press`, `unbrowse_select`, `unbrowse_scroll`, `unbrowse_submit`, `unbrowse_screenshot`, `unbrowse_text`, `unbrowse_markdown`, `unbrowse_cookies`, `unbrowse_eval`, `unbrowse_sync`, `unbrowse_close`
+
+Typical MCP host config:
+
+```json
+{
+  "mcpServers": {
+    "unbrowse": {
+      "command": "npx",
+      "args": ["-y", "unbrowse", "mcp"]
+    }
+  }
+}
+```
+
 ## Quick start
 
 ```bash
@@ -18,9 +47,13 @@ git clone --single-branch --depth 1 https://github.com/unbrowse-ai/unbrowse.git
 cd ~/unbrowse && ./setup --host off
 ```
 
-`./setup` installs repo dependencies, prebuilds the packaged CLI runtime, installs a stable `unbrowse` shim, and then runs the real first-time bootstrap: ToS acceptance, agent registration + API-key caching, and lobster.cash wallet detection when present.
+`./setup` installs repo dependencies, prebuilds the packaged CLI runtime, installs a stable `unbrowse` shim, and then runs the real first-time bootstrap: ToS acceptance, agent registration + API-key caching, and wallet detection when present.
 
-If a wallet is configured, that wallet address is synced onto your agent profile and becomes the destination for contributor payouts when your routes earn.
+If a wallet is configured, that wallet address becomes the contributor/payment truth: it is synced onto your agent profile, used as the destination for contributor payouts when your routes earn, and used as the spending wallet for paid marketplace routes.
+
+Recommended for new installs: set up Crossmint `lobster.cash` during bootstrap. `unbrowse setup` now encourages it, and when the tooling is already present it will try `npx @crossmint/lobster-cli setup` automatically. That wallet becomes the payout destination for contributed routes and the spending wallet for paid marketplace routes.
+
+Unbrowse supports wallet providers such as Crossmint `lobster.cash` for x402-gated routes. If you use `lobster.cash`, set `LOBSTER_WALLET_ADDRESS`. Other providers can use `AGENT_WALLET_ADDRESS` and optional `AGENT_WALLET_PROVIDER`.
 
 For repeat npm use after a healthy publish:
 
@@ -29,6 +62,15 @@ npm install -g unbrowse
 unbrowse setup
 ```
 
+For generic MCP hosts:
+
+```bash
+git clone --single-branch --depth 1 https://github.com/unbrowse-ai/unbrowse.git ~/unbrowse
+cd ~/unbrowse && ./setup --host mcp
+```
+
+That writes a ready-to-import MCP config to `~/.config/unbrowse/mcp/unbrowse.json`. A generic template is also published at [`/mcp.json`](https://www.unbrowse.ai/mcp.json).
+
 If your agent host uses skills:
 
 ```bash
@@ -39,6 +81,14 @@ npx skills add unbrowse-ai/unbrowse
 
 Unbrowse no longer self-updates at runtime. If you already have Unbrowse installed, upgrade to the latest version after each release or the new flow may not work on your machine.
 
+Check the exact command for your install with:
+
+```bash
+unbrowse upgrade
+```
+
+Codex and Claude installs now also get a session-start update hint during `unbrowse setup`, so newer releases are surfaced in the host before the CLI drifts too far behind.
+
 If you installed from a repo clone:
 
 ```bash
@@ -47,6 +97,14 @@ git pull --ff-only
 ./setup --host off
 ```
 
+If you installed for a generic MCP host:
+
+```bash
+cd ~/unbrowse
+git pull --ff-only
+./setup --host mcp
+```
+
 If your agent host uses skills, rerun its skill install/update command too:
 
 ```bash
@@ -55,30 +113,33 @@ npx skills add unbrowse-ai/unbrowse
 
 Need help or want release updates? Join the Discord: [discord.gg/VWugEeFNsG](https://discord.gg/VWugEeFNsG)
 
-Every CLI command auto-starts the local server on `http://localhost:6969` by default. Override with `UNBROWSE_URL`, `PORT`, or `HOST`. On first startup it auto-registers as an agent with the marketplace and caches credentials in `~/.unbrowse/config.json`. `unbrowse setup` prompts for an email-shaped identity first; headless setups can provide `UNBROWSE_AGENT_EMAIL`.
-Canonical docs: [docs.unbrowse.ai](https://docs.unbrowse.ai)
-
-Every CLI command auto-starts the local server on `http://localhost:6969` by default. Override with `UNBROWSE_URL`, `PORT`, or `HOST`. On first startup it auto-registers as an agent with the marketplace and caches credentials in `~/.unbrowse/config.json`. `unbrowse setup` now prompts for an email-shaped identity first; headless setups can provide `UNBROWSE_AGENT_EMAIL`.
+Every CLI command auto-starts the local runtime on `http://localhost:6969` by default, and `unbrowse mcp` uses that same runtime behind the MCP stdio surface. Override with `UNBROWSE_URL`, `PORT`, or `HOST`. On first startup it auto-registers as an agent with the marketplace and caches credentials in `~/.unbrowse/config.json`. `unbrowse setup` now prompts for an email-shaped identity first; headless setups can provide `UNBROWSE_AGENT_EMAIL`.
+Public companion docs: [docs.unbrowse.ai](https://docs.unbrowse.ai)
 
 Works with Claude Code, Open Code, Cursor, Codex, Windsurf, and any agent host that can call a local CLI or skill.
 
 ## What setup does
 
-- Checks local prerequisites for the npm/npx flow.
+- Checks the local runtime/package-manager environment for the repo bootstrap or packaged CLI path.
+- Prebuilds the packaged CLI runtime and installs the stable `unbrowse` shim for the repo bootstrap path.
 - Verifies the bundled Kuri binary, or builds it from the vendored Kuri source when working from repo source with Zig installed.
 - Registers the Open Code `/unbrowse` command when Open Code is present.
+- Runs the first-use flow: ToS, agent registration/API-key caching, wallet detection, and Crossmint `lobster.cash` encouragement.
 - Starts the local Unbrowse server unless `--no-start` is passed.
 
 ## Common commands
 
 ```bash
 unbrowse health
+unbrowse mcp
 unbrowse resolve --intent "get trending searches" --url "https://google.com" --pretty
 unbrowse login --url "https://calendar.google.com"
 unbrowse skills
 unbrowse search --intent "get stock prices"
 ```
 
+For most MCP hosts, the standard flow is `unbrowse_resolve` first, then `unbrowse_execute`. For JS-heavy or first-time capture workflows, use the browser tool chain: `unbrowse_go -> unbrowse_snap -> action tools -> unbrowse_submit/unbrowse_sync -> unbrowse_close`.
+
 ## Demo notes
 
 - First-time capture/indexing on a site can take 20-80 seconds. That is the slow path; repeats should be much faster.
@@ -91,12 +152,18 @@ If you tried Unbrowse on a site or API and could not get it to work, add it to [
 
 ## Docs
 
-The synced skill repo also carries the longer-form docs set:
+The synced skill repo also carries the public docs set:
+
+- [Quickstart](./docs/guides/quickstart.md)
+- [API reference](./docs/api.md)
+- [Deployment guide](./docs/deployment.md)
+- [Release checklist](./docs/RELEASING.md)
+
+Whitepaper companion docs:
 
 - [Whitepaper companion index](./docs/whitepaper/README.md)
 - [For Technical Readers](./docs/whitepaper/for-technical-readers.md)
 - [For Investors](./docs/whitepaper/for-investors.md)
-- [Analytics API](./docs/analytics-api.md)
 
 ## How it works
 

package/bin/unbrowse-update-hint.mjs

@@ -0,0 +1,22 @@
+#!/usr/bin/env node
+
+import { dirname, join } from "node:path";
+import { fileURLToPath } from "node:url";
+import { spawn } from "node:child_process";
+
+const __dirname = dirname(fileURLToPath(import.meta.url));
+const cliPath = join(__dirname, "..", "dist", "cli.js");
+
+const child = spawn(process.execPath, [cliPath, "upgrade", "--hint-only"], {
+  stdio: "inherit",
+  cwd: process.cwd(),
+  env: process.env,
+});
+
+child.on("exit", (code, signal) => {
+  if (signal) {
+    process.kill(process.pid, signal);
+    return;
+  }
+  process.exit(code ?? 0);
+});

package/bin/unbrowse-wrapper.mjs

@@ -5,35 +5,106 @@
  * falls back to the package-managed Node launcher if not.
  */
 
-import { existsSync } from "node:fs";
+import { existsSync, readFileSync } from "node:fs";
+import { createRequire } from "node:module";
 import { join, dirname } from "node:path";
 import { fileURLToPath } from "node:url";
 import { spawn } from "node:child_process";
 
 const __dirname = dirname(fileURLToPath(import.meta.url));
+const packageRoot = join(__dirname, "..");
 const binaryPath = join(__dirname, "unbrowse");
+const launcherPath = join(__dirname, "unbrowse.js");
+const require = createRequire(import.meta.url);
 
-if (existsSync(binaryPath)) {
-  // Compiled binary — exec directly, replace this process
-  const child = spawn(binaryPath, process.argv.slice(2), {
+const REQUIRED_FALLBACK_PACKAGES = [
+  "tsx",
+  "bs58",
+  "@solana/kit",
+  "@cascade-fyi/splits-sdk",
+];
+
+const KNOWN_BAD_FALLBACK_VERSIONS = new Set([
+  "2.10.2",
+]);
+
+function readInstalledVersion() {
+  try {
+    const pkg = JSON.parse(readFileSync(join(packageRoot, "package.json"), "utf8"));
+    return typeof pkg.version === "string" ? pkg.version : "unknown";
+  } catch {
+    return "unknown";
+  }
+}
+
+function printRepairHelp(reason) {
+  const installedVersion = readInstalledVersion();
+  const lines = [
+    `[unbrowse] ${reason}`,
+    `[unbrowse] Installed package version: ${installedVersion}`,
+    "[unbrowse] Repair: npm uninstall -g unbrowse && npm install -g unbrowse@latest",
+  ];
+  process.stderr.write(lines.join("\n") + "\n");
+}
+
+function failInstall(reason, exitCode = 1) {
+  printRepairHelp(reason);
+  process.exit(exitCode);
+}
+
+function missingFallbackPackages() {
+  const missing = [];
+  for (const specifier of REQUIRED_FALLBACK_PACKAGES) {
+    try {
+      require.resolve(specifier);
+    } catch {
+      missing.push(specifier);
+    }
+  }
+  return missing;
+}
+
+function spawnEntrypoint(command, args) {
+  const child = spawn(command, args, {
     stdio: "inherit",
+    cwd: process.cwd(),
     env: process.env,
   });
+  child.on("error", (error) => {
+    const details = error instanceof Error ? error.message : String(error);
+    if (error && typeof error === "object" && "code" in error && error.code === "EACCES") {
+      failInstall(`Launch target is not executable (${command}). Global install permissions are corrupted.`);
+    }
+    failInstall(`Failed to launch ${command}: ${details}`);
+  });
   child.on("exit", (code, signal) => {
     if (signal) { process.kill(process.pid, signal); return; }
     process.exit(code ?? 1);
   });
+}
+
+if (process.argv.includes("--version") || process.argv.includes("-v")) {
+  process.stdout.write(`${readInstalledVersion()}\n`);
+  process.exit(0);
+}
+
+if (existsSync(binaryPath)) {
+  // Compiled binary — exec directly, replace this process
+  spawnEntrypoint(binaryPath, process.argv.slice(2));
 } else {
+  const installedVersion = readInstalledVersion();
+  if (KNOWN_BAD_FALLBACK_VERSIONS.has(installedVersion)) {
+    failInstall(
+      `Installed package version ${installedVersion} is known-bad in source fallback mode and shipped an incomplete runtime dependency set.`,
+    );
+  }
+
+  const missing = missingFallbackPackages();
+  if (missing.length > 0) {
+    failInstall(`Fallback runtime is missing required packages: ${missing.join(", ")}.`);
+  }
+
   // Fallback: delegate to the stable package launcher so
   // npm installs and npx use the same dependency resolution path.
-  const launcherPath = join(__dirname, "unbrowse.js");
-  const child = spawn(process.execPath, [launcherPath, ...process.argv.slice(2)], {
-    stdio: "inherit",
-    cwd: process.cwd(),
-    env: process.env,
-  });
-  child.on("exit", (code, signal) => {
-    if (signal) { process.kill(process.pid, signal); return; }
-    process.exit(code ?? 1);
-  });
+  spawnEntrypoint(process.execPath, [launcherPath, ...process.argv.slice(2)]);
 }