unbrowse 2.12.4 → 2.12.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/README.md +10 -127
  2. package/dist/cli.js +493 -3442
  3. package/package.json +6 -15
  4. package/runtime-src/api/routes.ts +29 -1338
  5. package/runtime-src/auth/index.ts +33 -296
  6. package/runtime-src/capture/index.ts +123 -466
  7. package/runtime-src/cli.ts +374 -1288
  8. package/runtime-src/client/index.ts +30 -696
  9. package/runtime-src/execution/index.ts +143 -694
  10. package/runtime-src/graph/index.ts +10 -128
  11. package/runtime-src/intent-match.ts +27 -27
  12. package/runtime-src/kuri/client.ts +242 -1210
  13. package/runtime-src/orchestrator/index.ts +499 -2753
  14. package/runtime-src/reverse-engineer/index.ts +22 -239
  15. package/runtime-src/runtime/local-server.ts +22 -238
  16. package/runtime-src/runtime/paths.ts +5 -9
  17. package/runtime-src/runtime/setup.ts +1 -56
  18. package/runtime-src/server.ts +11 -11
  19. package/runtime-src/transform/schema-hints.ts +358 -0
  20. package/runtime-src/types/skill.ts +2 -488
  21. package/runtime-src/verification/index.ts +14 -35
  22. package/runtime-src/version.ts +8 -68
  23. package/vendor/kuri/darwin-arm64/kuri +0 -0
  24. package/vendor/kuri/darwin-x64/kuri +0 -0
  25. package/vendor/kuri/linux-arm64/kuri +0 -0
  26. package/vendor/kuri/linux-x64/kuri +0 -0
  27. package/SKILL.md +0 -754
  28. package/bin/unbrowse-update-hint.mjs +0 -22
  29. package/bin/unbrowse-wrapper.mjs +0 -107
  30. package/bin/unbrowse.js +0 -37
  31. package/dist/mcp.js +0 -1796
  32. package/runtime-src/agent-outcome.ts +0 -166
  33. package/runtime-src/analytics-session.ts +0 -55
  34. package/runtime-src/api/browse-index.ts +0 -254
  35. package/runtime-src/api/browse-session.ts +0 -648
  36. package/runtime-src/api/browse-submit-prereqs.ts +0 -48
  37. package/runtime-src/api/browse-submit.ts +0 -1184
  38. package/runtime-src/auth/runtime.ts +0 -116
  39. package/runtime-src/browser/index.ts +0 -643
  40. package/runtime-src/browser/types.ts +0 -41
  41. package/runtime-src/build-info.generated.ts +0 -4
  42. package/runtime-src/capture/prefetch.ts +0 -122
  43. package/runtime-src/capture/rsc.ts +0 -45
  44. package/runtime-src/cli/shortcuts.ts +0 -273
  45. package/runtime-src/client/graph-client.ts +0 -99
  46. package/runtime-src/execution/robots.ts +0 -167
  47. package/runtime-src/execution/search-forms.ts +0 -188
  48. package/runtime-src/graph/planner.ts +0 -411
  49. package/runtime-src/graph/session.ts +0 -294
  50. package/runtime-src/graph/trace-store.ts +0 -136
  51. package/runtime-src/indexer/index.ts +0 -441
  52. package/runtime-src/mcp.ts +0 -1522
  53. package/runtime-src/orchestrator/browser-agent.ts +0 -374
  54. package/runtime-src/orchestrator/dag-advisor.ts +0 -59
  55. package/runtime-src/orchestrator/dag-feedback.ts +0 -256
  56. package/runtime-src/orchestrator/first-pass-action.ts +0 -403
  57. package/runtime-src/orchestrator/passive-publish.ts +0 -182
  58. package/runtime-src/orchestrator/timing-economics.ts +0 -80
  59. package/runtime-src/payments/cascade.ts +0 -137
  60. package/runtime-src/payments/index.ts +0 -268
  61. package/runtime-src/payments/wallet.ts +0 -98
  62. package/runtime-src/publish/sanitize.ts +0 -197
  63. package/runtime-src/publish-admission.ts +0 -279
  64. package/runtime-src/reverse-engineer/description-prompt.ts +0 -213
  65. package/runtime-src/router.ts +0 -17
  66. package/runtime-src/routing-telemetry.ts +0 -395
  67. package/runtime-src/runtime/browser-access.ts +0 -11
  68. package/runtime-src/runtime/browser-auth.ts +0 -12
  69. package/runtime-src/runtime/browser-host.ts +0 -48
  70. package/runtime-src/runtime/lifecycle.ts +0 -17
  71. package/runtime-src/runtime/supervisor.ts +0 -69
  72. package/runtime-src/runtime/update-hints.ts +0 -351
  73. package/runtime-src/settings.ts +0 -221
  74. package/runtime-src/single-binary.ts +0 -141
  75. package/runtime-src/site-policy.ts +0 -54
  76. package/runtime-src/stale-cleanup-runner.ts +0 -144
  77. package/runtime-src/stale-cleanup.ts +0 -133
  78. package/runtime-src/telemetry-attribution.ts +0 -120
  79. package/runtime-src/telemetry.ts +0 -253
  80. package/runtime-src/verification/auth-gate.ts +0 -8
  81. package/runtime-src/verification/candidates.ts +0 -27
  82. package/runtime-src/verification/matrix.ts +0 -30
  83. package/runtime-src/workflow/artifact.ts +0 -161
  84. package/runtime-src/workflow/compile.ts +0 -808
  85. package/runtime-src/workflow/publish.ts +0 -205
  86. package/runtime-src/workflow/runtime.ts +0 -213
  87. package/scripts/postinstall.mjs +0 -105
  88. package/scripts/release-assets.mjs +0 -24
  89. package/scripts/verify-release-assets.mjs +0 -39
  90. package/vendor/kuri/manifest.json +0 -24
package/README.md CHANGED
@@ -2,85 +2,28 @@
2
2
 
3
3
  This package installs the `unbrowse` CLI.
4
4
 
5
- Unbrowse is a local Model Context Protocol (MCP) server and CLI that turns any website into a reusable API interface for agents. It captures network traffic, reverse-engineers the real endpoints underneath the UI, and stores what it learns in a shared marketplace so the next agent can reuse it instantly.
5
+ Turn any website into a reusable API interface for agents. Unbrowse captures network traffic, reverse-engineers the real endpoints underneath the UI, and stores what it learns in a shared marketplace so the next agent can reuse it instantly.
6
6
 
7
7
  One agent learns a site once. Every later agent gets the fast path.
8
8
 
9
- Unbrowse is a drop-in replacement for OpenClaw / `agent-browser` browser flows for agents: on the API-native path it is typically ~30x faster, ~90% cheaper, and turns repeated browser work into reusable route assets.
10
-
11
9
  > Security note: capture and execution stay local by default. Credentials stay on your machine. Learned API contracts are published to the shared marketplace only after capture. See [SKILL.md](./SKILL.md) for the full agent-facing API reference and tool-policy guidance.
12
10
 
13
- ## MCP server
14
-
15
- Unbrowse implements the Model Context Protocol over stdio. `unbrowse mcp` is the MCP server entrypoint.
16
-
17
- - Protocol: JSON-RPC 2.0 MCP over stdio
18
- - Handshake: `initialize`, `notifications/initialized`, `ping`
19
- - Capability surface today: `tools/list` and `tools/call`
20
- - Current MCP shape: tools only. No `resources/*` or `prompts/*` methods yet.
21
- - Runtime model: the MCP server fronts the local Unbrowse runtime on `http://localhost:6969`; hosts talk standard MCP, and Unbrowse uses the local HTTP runtime behind the scenes.
22
-
23
- Core MCP tools:
24
-
25
- - Discovery: `unbrowse_health`, `unbrowse_search`, `unbrowse_resolve`, `unbrowse_execute`, `unbrowse_feedback`
26
- - Auth/cache: `unbrowse_login`, `unbrowse_skills`, `unbrowse_skill`, `unbrowse_sessions`
27
- - Browser capture: `unbrowse_go`, `unbrowse_snap`, `unbrowse_click`, `unbrowse_fill`, `unbrowse_type`, `unbrowse_press`, `unbrowse_select`, `unbrowse_scroll`, `unbrowse_submit`, `unbrowse_screenshot`, `unbrowse_text`, `unbrowse_markdown`, `unbrowse_cookies`, `unbrowse_eval`, `unbrowse_sync`, `unbrowse_close`
28
-
29
- Typical MCP host config:
30
-
31
- ```json
32
- {
33
- "mcpServers": {
34
- "unbrowse": {
35
- "command": "npx",
36
- "args": ["-y", "unbrowse", "mcp"]
37
- }
38
- }
39
- }
40
- ```
41
-
42
11
  ## Quick start
43
12
 
44
13
  ```bash
45
- # One-line install from the latest GitHub release
46
- curl -fsSL https://unbrowse.ai/install.sh | sh
14
+ # Fastest full setup
15
+ npx unbrowse setup
47
16
  ```
48
17
 
49
- That installer now follows the Kuri pattern: detect platform, download the matching release tarball, install `unbrowse` into `~/.local/bin`, then run `unbrowse setup`.
50
-
51
- ```bash
52
- # Deterministic setup from a repo clone
53
- git clone --single-branch --depth 1 https://github.com/unbrowse-ai/unbrowse.git ~/unbrowse
54
- cd ~/unbrowse && ./setup --host off
55
- ```
18
+ `npx unbrowse setup` downloads the CLI on demand, verifies the bundled Kuri runtime, lets you register with an email-shaped display identity, registers the Open Code `/unbrowse` command when Open Code is detected, and starts the local server.
56
19
 
57
- `./setup` installs repo dependencies, prebuilds the packaged CLI runtime, installs a stable `unbrowse` shim, and then runs the real first-time bootstrap: ToS acceptance, agent registration + API-key caching, and wallet detection when present.
58
-
59
- If a wallet is configured, that wallet address becomes the contributor/payment truth: it is synced onto your agent profile, used as the destination for contributor payouts when your routes earn, and used as the spending wallet for paid marketplace routes.
60
-
61
- Recommended for new installs: set up Crossmint `lobster.cash` during bootstrap. `unbrowse setup` now encourages it, and when the tooling is already present it will try `npx @crossmint/lobster-cli setup` automatically. That wallet becomes the payout destination for contributed routes and the spending wallet for paid marketplace routes.
62
-
63
- Unbrowse supports wallet providers such as Crossmint `lobster.cash` for x402-gated routes. If you use `lobster.cash`, set `LOBSTER_WALLET_ADDRESS`. Other providers can use `AGENT_WALLET_ADDRESS` and optional `AGENT_WALLET_PROVIDER`.
64
-
65
- For repeat npm use after a healthy publish:
20
+ For daily use:
66
21
 
67
22
  ```bash
68
23
  npm install -g unbrowse
69
24
  unbrowse setup
70
25
  ```
71
26
 
72
- The npm package is binary-first: install downloads the prebuilt Bun-compiled CLI for your platform. It does not ship the TypeScript runtime or JS fallback launcher.
73
- On supported platforms, install now fails fast if the matching GitHub release asset is missing or unreachable.
74
-
75
- For generic MCP hosts:
76
-
77
- ```bash
78
- git clone --single-branch --depth 1 https://github.com/unbrowse-ai/unbrowse.git ~/unbrowse
79
- cd ~/unbrowse && ./setup --host mcp
80
- ```
81
-
82
- That writes a ready-to-import MCP config to `~/.config/unbrowse/mcp/unbrowse.json`. A generic template is also published at [`/mcp.json`](https://www.unbrowse.ai/mcp.json).
83
-
84
27
  If your agent host uses skills:
85
28
 
86
29
  ```bash
@@ -91,28 +34,11 @@ npx skills add unbrowse-ai/unbrowse
91
34
 
92
35
  Unbrowse no longer self-updates at runtime. If you already have Unbrowse installed, upgrade to the latest version after each release or the new flow may not work on your machine.
93
36
 
94
- Check the exact command for your install with:
37
+ If you installed the CLI globally:
95
38
 
96
39
  ```bash
97
- unbrowse upgrade
98
- ```
99
-
100
- Codex and Claude installs now also get a session-start update hint during `unbrowse setup`, so newer releases are surfaced in the host before the CLI drifts too far behind.
101
-
102
- If you installed from a repo clone:
103
-
104
- ```bash
105
- cd ~/unbrowse
106
- git pull --ff-only
107
- ./setup --host off
108
- ```
109
-
110
- If you installed for a generic MCP host:
111
-
112
- ```bash
113
- cd ~/unbrowse
114
- git pull --ff-only
115
- ./setup --host mcp
40
+ npm install -g unbrowse@latest
41
+ unbrowse setup
116
42
  ```
117
43
 
118
44
  If your agent host uses skills, rerun its skill install/update command too:
@@ -123,44 +49,27 @@ npx skills add unbrowse-ai/unbrowse
123
49
 
124
50
  Need help or want release updates? Join the Discord: [discord.gg/VWugEeFNsG](https://discord.gg/VWugEeFNsG)
125
51
 
126
- Every CLI command auto-starts the local runtime on `http://localhost:6969` by default, and `unbrowse mcp` uses that same runtime behind the MCP stdio surface. Override with `UNBROWSE_URL`, `PORT`, or `HOST`. On first startup it auto-registers as an agent with the marketplace and caches credentials in `~/.unbrowse/config.json`. `unbrowse setup` now prompts for an email-shaped identity first; headless setups can provide `UNBROWSE_AGENT_EMAIL`.
127
- Public companion docs: [docs.unbrowse.ai](https://docs.unbrowse.ai)
52
+ Every CLI command auto-starts the local server on `http://localhost:6969` by default. Override with `UNBROWSE_URL`, `PORT`, or `HOST`. On first startup it auto-registers as an agent with the marketplace and caches credentials in `~/.unbrowse/config.json`. `unbrowse setup` now prompts for an email-shaped identity first; headless setups can provide `UNBROWSE_AGENT_EMAIL`.
128
53
 
129
54
  Works with Claude Code, Open Code, Cursor, Codex, Windsurf, and any agent host that can call a local CLI or skill.
130
55
 
131
56
  ## What setup does
132
57
 
133
- - Checks the local runtime/package-manager environment for the repo bootstrap or packaged CLI path.
134
- - Prebuilds the packaged CLI runtime and installs the stable `unbrowse` shim for the repo bootstrap path.
58
+ - Checks local prerequisites for the npm/npx flow.
135
59
  - Verifies the bundled Kuri binary, or builds it from the vendored Kuri source when working from repo source with Zig installed.
136
60
  - Registers the Open Code `/unbrowse` command when Open Code is present.
137
- - Runs the first-use flow: ToS, agent registration/API-key caching, wallet detection, and Crossmint `lobster.cash` encouragement.
138
61
  - Starts the local Unbrowse server unless `--no-start` is passed.
139
62
 
140
63
  ## Common commands
141
64
 
142
65
  ```bash
143
66
  unbrowse health
144
- unbrowse mcp
145
67
  unbrowse resolve --intent "get trending searches" --url "https://google.com" --pretty
146
68
  unbrowse login --url "https://calendar.google.com"
147
69
  unbrowse skills
148
70
  unbrowse search --intent "get stock prices"
149
71
  ```
150
72
 
151
- For most MCP hosts, the standard flow is `unbrowse_resolve` first, then `unbrowse_execute`. For JS-heavy or first-time capture workflows, use the browser tool chain: `unbrowse_go -> unbrowse_snap -> action tools -> unbrowse_submit/unbrowse_sync -> unbrowse_close`.
152
-
153
- ## Dependency walk for multi-step UIs
154
-
155
- Treat each successful browser submit as a dependency boundary.
156
-
157
- - Do not jump straight to guessed downstream URLs like `/date-selection.html` or `/payment.html` unless the current session already reached them through the real page flow.
158
- - Use `unbrowse_submit` for the actual transition, then trust the returned `url`, `session_id`, and any next-step hints over your own assumptions.
159
- - `unbrowse_sync` after a good transition so the route graph records which request chain unlocked the next page.
160
- - If a page later returns `abandonedCart`, `session_expired`, or a wrong audience/product variant, restart from the last known good upstream step and walk forward again.
161
-
162
- The dependency graph is not just API-to-API. On JS-heavy checkout flows it also captures browser-state prerequisites: selected product, resident/non-resident audience, date, slot, auth, and cart state. Future agents should reason from those prerequisites before calling deeper steps.
163
-
164
73
  ## Demo notes
165
74
 
166
75
  - First-time capture/indexing on a site can take 20-80 seconds. That is the slow path; repeats should be much faster.
@@ -171,21 +80,6 @@ The dependency graph is not just API-to-API. On JS-heavy checkout flows it also
171
80
 
172
81
  If you tried Unbrowse on a site or API and could not get it to work, add it to [Discussion #53](https://github.com/unbrowse-ai/unbrowse/discussions/53). We use that thread to collect missing or broken targets so we can turn them into requirements for the next eval pass.
173
82
 
174
- ## Docs
175
-
176
- The synced skill repo also carries the public docs set:
177
-
178
- - [Quickstart](./docs/guides/quickstart.md)
179
- - [API reference](./docs/api.md)
180
- - [Deployment guide](./docs/deployment.md)
181
- - [Release checklist](./docs/RELEASING.md)
182
-
183
- Whitepaper companion docs:
184
-
185
- - [Whitepaper companion index](./docs/whitepaper/README.md)
186
- - [For Technical Readers](./docs/whitepaper/for-technical-readers.md)
187
- - [For Investors](./docs/whitepaper/for-investors.md)
188
-
189
83
  ## How it works
190
84
 
191
85
  When an agent asks for something, Unbrowse first searches the marketplace for an existing skill. If one exists with enough confidence, it executes immediately. If not, Unbrowse captures the site, learns the APIs behind it, publishes a reusable skill, and executes that instead.
@@ -241,7 +135,6 @@ Non-GET endpoints (POST, PUT, DELETE) require explicit confirmation:
241
135
 
242
136
  - `dry_run: true` — preview what would execute without side effects
243
137
  - `confirm_unsafe: true` — explicit user consent to proceed
244
- - `confirm_third_party_terms: true` — extra explicit confirmation for policy-sensitive domains/actions such as X write endpoints
245
138
 
246
139
  GET endpoints auto-execute. Mutations never fire without opt-in.
247
140
 
@@ -264,16 +157,6 @@ See [SKILL.md](./SKILL.md) for the full API reference including all endpoints, s
264
157
  | GET | `/v1/stats/summary` | Platform stats |
265
158
  | GET | `/health` | Health check |
266
159
 
267
- ## Docs
268
-
269
- The standalone skill repo also carries the core repo docs:
270
-
271
- - [Quickstart guide](./docs/guides/quickstart.md)
272
- - [API notes](./docs/api.md)
273
- - [Codex eval harness](./docs/codex-eval-harness.md)
274
- - [Deployment notes](./docs/deployment.md)
275
- - [Release checklist](./docs/RELEASING.md)
276
-
277
160
  ## Configuration
278
161
 
279
162
  ### Runtime directories