unbrowse 6.17.2 → 7.0.0

package/README.md

@@ -1,8 +1,6 @@
 # Unbrowse
 
-> ## ⚠️ The Anthropic skill path retired in v6.15.0
->
-> Agents should connect via the **MCP server**. The CLI (`unbrowse ...`) and SDK are unaffected. Add this to your MCP host config (Claude Desktop, Cursor, Codex, or any MCP-compatible client):
+> Skill, MCP server, SDK, CLI — four ways to use the same local runtime. `SKILL.md` is back in this package; the MCP block below still works as the canonical agent path. Pick what your host supports.
 >
 > ```json
 > {
@@ -15,17 +13,17 @@
 > }
 > ```
 >
-> Then run `npx unbrowse setup` once. The `SKILL.md` file from this package was retired in v6.15.0; the MCP block above replaces it.
+> Then run `npx unbrowse setup` once. The SKILL.md in this directory is loadable via `find-skills` / `/find-skills`; reading it gives an agent the same map (resolve + execute + browse-session) without an MCP host.
 
 This package installs the `unbrowse` CLI.
 
-Unbrowse is a local Model Context Protocol (MCP) server and CLI that turns any website into a reusable API interface for agents. It captures network traffic, reverse-engineers the real endpoints underneath the UI, and stores what it learns in a shared marketplace so the next agent can reuse it instantly.
+Unbrowse is a local Model Context Protocol (MCP) server and CLI that turns websites into reusable API routes for agents. It learns callable routes from explicit browsing sessions, keeps credentials local, and shares only sanitized route metadata when you publish.
 
 One agent learns a site once. Every later agent gets the fast path.
 
 Unbrowse is a drop-in replacement for OpenClaw / `agent-browser` browser flows for agents: on the API-native path it is typically ~30x faster, ~90% cheaper, and turns repeated browser work into reusable route assets.
 
-> Security note: capture and execution stay local by default. Credentials stay on your machine. Learned API contracts are published to the shared marketplace only after capture.
+> Security note: capture and execution stay local by default. Credentials stay on your machine. Learned routes are published to the shared marketplace only after an explicit publish checkpoint.
 ## MCP server
 
 Unbrowse implements the Model Context Protocol over stdio. `unbrowse mcp` is the MCP server entrypoint.
@@ -65,12 +63,12 @@ curl -fsSL https://unbrowse.ai/install.sh | sh
 That installer now follows the Kuri pattern: detect platform, download the matching release tarball, install `unbrowse` into `~/.local/bin`, then run `unbrowse setup`.
 
 ```bash
-# Deterministic setup from a repo clone
-git clone --single-branch --depth 1 https://github.com/unbrowse-ai/unbrowse.git ~/unbrowse
-cd ~/unbrowse && ./setup --host off
+# Deterministic npm setup
+npm install -g unbrowse
+unbrowse setup
 ```
 
-`./setup` installs repo dependencies, prebuilds the packaged CLI runtime, installs a stable `unbrowse` shim, and then runs the real first-time bootstrap: ToS acceptance, agent registration + API-key caching, and wallet detection when present.
+`unbrowse setup` runs the first-time bootstrap: ToS acceptance, agent registration + API-key caching, and wallet detection when present.
 
 If a wallet is configured, that wallet address becomes the contributor/payment truth: it is synced onto your agent profile, used as the destination for contributor payouts when your routes earn, and used as the spending wallet for paid marketplace routes.
 
@@ -91,8 +89,7 @@ On supported platforms, install now fails fast if the matching GitHub release as
 For generic MCP hosts:
 
 ```bash
-git clone --single-branch --depth 1 https://github.com/unbrowse-ai/unbrowse.git ~/unbrowse
-cd ~/unbrowse && ./setup --host mcp
+npx unbrowse setup --mcp
 ```
 
 That writes a ready-to-import MCP config to `~/.config/unbrowse/mcp/unbrowse.json`. A generic template is also published at [`/mcp.json`](https://www.unbrowse.ai/mcp.json).
@@ -100,7 +97,7 @@ That writes a ready-to-import MCP config to `~/.config/unbrowse/mcp/unbrowse.jso
 If your agent host uses skills:
 
 ```bash
-npx skills add unbrowse-ai/unbrowse
+npm install -g unbrowse
 ```
 
 ## Upgrading
@@ -181,7 +178,7 @@ The dependency graph is not just API-to-API. On JS-heavy checkout flows it also
 
 - First-time capture/indexing on a site can take 20-80 seconds. That is the slow path; repeats should be much faster.
 - For website tasks, keep the agent on Unbrowse instead of letting it drift into generic web search or ad hoc `curl`.
-- Reddit is still a harder target than most sites because of anti-bot protections. Prefer canonical `.json` routes when available.
+- Some sites are harder targets than others. Prefer documented JSON routes when a site offers them.
 
 ## Help shape the next eval
 
@@ -191,20 +188,20 @@ If you tried Unbrowse on a site or API and could not get it to work, add it to [
 
 The synced skill repo also carries the public docs set:
 
-- [Quickstart](./docs/guides/quickstart.md)
-- [API reference](./docs/api.md)
-- [Deployment guide](./docs/deployment.md)
-- [Release checklist](./docs/RELEASING.md)
+- [Quickstart](https://docs.unbrowse.ai/guides/quickstart)
+- [Agent workflow](https://docs.unbrowse.ai/for-agents/how-an-agent-uses-unbrowse)
+- [Integration surfaces](https://docs.unbrowse.ai/for-developers/integration-surfaces)
+- [Payment model](https://docs.unbrowse.ai/HOW_UNBROWSE_PAYS)
 
 Whitepaper companion docs:
 
-- [Whitepaper companion index](./docs/whitepaper/README.md)
-- [For Technical Readers](./docs/whitepaper/for-technical-readers.md)
-- [For Investors](./docs/whitepaper/for-investors.md)
+- [Whitepaper companion index](https://docs.unbrowse.ai/whitepaper/)
+- [For Technical Readers](https://docs.unbrowse.ai/whitepaper/for-technical-readers)
+- [For Investors](https://docs.unbrowse.ai/whitepaper/for-investors)
 
 ## How it works
 
-When an agent asks for something, Unbrowse first searches the marketplace for an existing skill. If one exists with enough confidence, it executes immediately. If not, Unbrowse captures the site, learns the APIs behind it, publishes a reusable skill, and executes that instead.
+When an agent asks for something, Unbrowse first searches the marketplace for an existing skill. If one exists with enough confidence, it executes immediately. If not, Unbrowse can open a local browser session, learn reusable route metadata, and publish it only after the configured checkpoint.
 
 Every learned skill becomes discoverable by every future agent. Reliability scoring, feedback, schema drift, and verification keep the good paths hot and the broken ones out of the way.
 
@@ -214,7 +211,7 @@ When you call `POST /v1/intent/resolve`, the orchestrator follows this priority
 
 1. **Route cache** (5-min TTL) — instant hit if the same intent was recently resolved
 2. **Marketplace search** — semantic vector search ranked by composite score: 40% embedding similarity + 30% reliability + 15% freshness + 15% verification status
-3. **Live capture** — headless browser records network traffic, reverse-engineers API endpoints, publishes a new skill
+3. **Local browser session** — the runtime observes allowed requests, indexes reusable route metadata, and can publish a new skill after review
 4. **DOM fallback** — if no API endpoints are found (static/SSR sites), structured data is extracted from rendered HTML
 
 Skills published by live capture become available to all agents on the network.
@@ -241,15 +238,15 @@ A background verification loop runs every 6 hours, executing safe (GET) endpoint
 
 ## Authentication for gated sites
 
-For most sites, auth is automatic. If you're logged into a site in Chrome or Firefox, Unbrowse reads your cookies directly from the browser's SQLite database — no extra steps needed. Cookies are resolved fresh on every call, so sessions stay current. For Chromium-family apps and Electron shells, `/v1/auth/steal` also accepts a custom cookie DB path or user-data dir plus an optional macOS Safe Storage service name.
+For most sites, sign-in works from your existing local browser session or from an interactive login window. Sessions stay on your machine and are reused only by your local runtime.
 
 | Strategy            | How it works                                       | When to use                                          |
 | ------------------- | -------------------------------------------------- | ---------------------------------------------------- |
-| Auto cookie resolve | Reads cookie DBs from Chrome/Firefox automatically | Default — works if you're logged in via your browser |
+| Local session reuse | Uses your existing local browser session | Default — works if you're logged in via your browser |
 | Yolo mode           | Opens Chrome with your real profile                | Sites with complex auth (OAuth popups, 2FA)          |
 | Interactive login   | Opens a headed browser for manual login            | Fallback when auto-resolve has no cookies            |
 
-Auth headers (CSRF tokens, API keys, authorization headers) are captured during browsing and stored in an encrypted vault (`~/.unbrowse/vault/`). Server-side fetches replay these headers automatically — no browser launch needed. Cross-domain auth (e.g. lu.ma cookies working on api2.luma.com) is handled transparently. Stale credentials (401/403 responses) are auto-deleted.
+Authentication material stays local, is stored encrypted, and is automatically refreshed or discarded when a site rejects it. The marketplace receives route metadata, not your private session.
 
 ## Mutation safety
 
@@ -270,7 +267,7 @@ See the public API reference below for endpoints, search, feedback, auth, and is
 | POST   | `/v1/intent/resolve`     | Search marketplace, capture if needed, execute |
 | POST   | `/v1/skills/:id/execute` | Execute a specific skill                       |
 | POST   | `/v1/auth/login`         | Interactive browser login                      |
-| POST   | `/v1/auth/steal`         | Import cookies from browser/Electron storage   |
+| POST   | `/v1/auth/import`        | Import a local browser session                 |
 | POST   | `/v1/search`             | Semantic search across all domains             |
 | POST   | `/v1/search/domain`      | Semantic search scoped to a domain             |
 | POST   | `/v1/feedback`           | Submit feedback (affects reliability scores)   |
@@ -284,11 +281,10 @@ See the public API reference below for endpoints, search, feedback, auth, and is
 
 The standalone skill repo also carries the core repo docs:
 
-- [Quickstart guide](./docs/guides/quickstart.md)
-- [API notes](./docs/api.md)
-- [Codex eval harness](./docs/codex-eval-harness.md)
-- [Deployment notes](./docs/deployment.md)
-- [Release checklist](./docs/RELEASING.md)
+- [Quickstart guide](https://docs.unbrowse.ai/guides/quickstart)
+- [Agent workflow](https://docs.unbrowse.ai/for-agents/how-an-agent-uses-unbrowse)
+- [Integration surfaces](https://docs.unbrowse.ai/for-developers/integration-surfaces)
+- [Payment model](https://docs.unbrowse.ai/HOW_UNBROWSE_PAYS)
 
 ## Configuration
 
@@ -296,8 +292,7 @@ The standalone skill repo also carries the core repo docs:
 
 ```
 ~/.unbrowse/config.json                # API key, agent ID, registration
-~/.unbrowse/vault/credentials.enc      # Encrypted credential store
-~/.unbrowse/vault/.key                 # Encryption key (mode 0o600)
+~/.unbrowse/vault/                     # Encrypted local credential store
 ~/.unbrowse/skill-cache/               # Local skill manifest cache
 ~/.unbrowse/profiles/<domain>/         # Per-domain Chrome profiles
 ~/.unbrowse/logs/unbrowse-YYYY-MM-DD.log  # Daily logs
@@ -323,13 +318,13 @@ src/
 ├── api/routes.ts         # HTTP route definitions
 ├── orchestrator/         # Intent resolution pipeline
 ├── execution/            # Skill/endpoint execution + retry logic
-├── capture/              # Headless browser traffic recording
-├── reverse-engineer/     # HAR parsing → endpoint extraction
+├── capture/              # Local browser session recording
+├── route-indexing/       # Captured requests → reusable route metadata
 ├── extraction/           # DOM structured data extraction
 ├── marketplace/          # Backend API client (beta-api.unbrowse.ai)
 ├── client/               # Agent registration & config management
-├── auth/                 # Interactive login + cookie extraction
-├── vault/                # Encrypted credential storage (AES-256-CBC)
+├── auth/                 # Interactive login + local session reuse
+├── vault/                # Encrypted credential storage
 ├── transform/            # Field projection + schema drift detection
 ├── verification/         # Periodic endpoint health checks
 ├── ratelimit/            # Request throttling

package/SKILL.md

@@ -0,0 +1,93 @@
+---
+name: unbrowse
+description: Capture once, replay everywhere — Unbrowse learns reusable route metadata from allowed browsing sessions and replays it as a fast, cheap, indexed route. Brings back the skill surface alongside the MCP server + SDK + CLI.
+metadata:
+  type: integration
+  origin: unbrowse-ai/unbrowse
+---
+
+# Unbrowse
+
+Unbrowse turns websites into reusable API routes for agents. Teach a route once, store sanitized metadata, replay it on later calls. Typical run is 30× faster and 90× cheaper than a fresh browser session.
+
+Three surfaces, one runtime:
+
+| Surface | When to reach for it |
+|---|---|
+| **MCP server** | An MCP-host agent (Claude Desktop, Cursor, Codex, Claude Code). Tool calls like `unbrowse_resolve`, `unbrowse_execute`, `unbrowse_go` appear in the host. |
+| **CLI** (`unbrowse`) | A shell session or a bash-script that wants the same surface as the MCP server, without an MCP host. |
+| **SDK** (`@unbrowse/sdk`) | A TypeScript program that wants to embed Unbrowse. `npm i @unbrowse/sdk` is enough; the SDK spawns its own local binary. |
+
+All three resolve to the same runtime workflow underneath:
+- **resolve** asks "is there an indexed route for this intent + URL?" — returns a shortlist or a hard handoff.
+- **execute** picks one endpoint from the shortlist and runs it — returns the real data.
+- **browse-session** opens a managed browser when the API is too dynamic to predict; local capture indexes route metadata.
+
+The two-tool flow (resolve + execute) is the agent UX north star: never one call, never three. The shortlist is structured so the calling LLM picks; execute is paid via x402 when the route is priced.
+
+## Quickstart
+
+For an MCP host (Claude Desktop, Cursor, Claude Code, Codex):
+
+```json
+{
+  "mcpServers": {
+    "unbrowse": {
+      "command": "npx",
+      "args": ["-y", "unbrowse", "mcp"]
+    }
+  }
+}
+```
+
+Then once:
+
+```bash
+npx unbrowse setup
+```
+
+For a shell:
+
+```bash
+unbrowse resolve --intent "search hacker news for openai" --url https://news.ycombinator.com
+unbrowse execute --skill-id <id-from-resolve> --endpoint-id <id-from-shortlist>
+```
+
+For a Node program:
+
+```bash
+npm i @unbrowse/sdk
+```
+
+```typescript
+import { spawn } from '@unbrowse/sdk';
+const client = await spawn();
+const resolved = await client.resolve({ intent: 'search hn for openai', url: 'https://news.ycombinator.com' });
+const result = await client.execute({ skillId: resolved.skill.id, endpointId: resolved.endpoints[0].id });
+```
+
+## /contract-shape (why three surfaces stay coherent)
+
+Every tool call on every surface follows the same shape:
+
+1. **Declare** — the call carries an intent + URL.
+2. **Iterate** — the runtime tries cached → marketplace → first-pass browser → live capture, in that order. Each step appends one trace row.
+3. **Mark with proof** — the response carries a trace with `success`, `skill_id`, `endpoint_id`. The proof is the trace row.
+
+Honest residue surfaces as a `next_step` field (`open_browse_session`, `abandon_or_authenticate`) instead of a one-word error.
+
+## What lives in the public docs
+
+Every primitive Unbrowse depends on (pointer-not-payload, residential proxy fallback, interstitial shortcut, x402+Faremeter, never-leaked-fields list, domain opt-out, fair split + claim, deploy gate, dimensional bench, kuri first-principles roadmap) is documented at [`docs/public/primitives/`](../../docs/public/primitives/) in the public repo. The README index there is enforced by `scripts/check-primitives-doc-public.sh` so the folder cannot drift from the codebase.
+
+## Skill / MCP / SDK / CLI sync (the precommit gate)
+
+When a shipping-surface signal changes (new top-level dir, new workspace member, new binary, new wrangler.toml target, new deploy workflow), the same commit must update a canonical doc. The gate at `scripts/precommit-doc-delta.sh` surfaces the delta as evidence on every iterate; full canonical wiring at `~/.claude/skills/meta-harness/scripts/gates/doc-delta.sh`.
+
+The skill surface (this file) updates whenever the MCP tool catalog or the SDK API changes. The precommit gate flags it; the agent ships the update in the same commit.
+
+## Provenance
+
+Source code: <https://github.com/unbrowse-ai/unbrowse-dev>  
+Public mirror: <https://github.com/unbrowse-ai/unbrowse>  
+MCP server, CLI, SDK published from this monorepo. Backend (`backend/`) is the Cloudflare Worker that handles marketplace + sponsor tier; frontend (`frontend/`) is the landing page; `packages/skill/` is this package (the npm-published CLI binary + the skill manifest you're reading).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "unbrowse",
-  "version": "6.17.2",
+  "version": "7.0.0",
   "description": "Reverse-engineer any website into reusable API skills. Zero-dep single binary with embedded browser engine.",
   "type": "module",
   "bin": {
@@ -14,6 +14,7 @@
     "scripts/postinstall-ping.mjs",
     "scripts/verify-release-assets.mjs",
     "README.md",
+    "SKILL.md",
     "LICENSE"
   ],
   "scripts": {

package/vendor/kuri/manifest.json

@@ -1,8 +1,8 @@
 {
-  "repo_url": "https://github.com/lekt9/kuri.git",
-  "branch": "feat/windows-port-wave-1",
+  "repo_url": "https://github.com/justrach/kuri.git",
+  "branch": "adding-extensions",
   "source_sha": "3cdc33c8507bd91f1ce499a69f567189e1284e4e",
-  "built_at": "2026-05-23T13:52:07.751Z",
+  "built_at": "2026-05-22T07:18:34.843Z",
   "binaries": {
     "darwin-arm64": {
       "zig_target": "aarch64-macos",