unbrowse 2.12.4 → 2.12.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -127
- package/dist/cli.js +493 -3442
- package/package.json +6 -15
- package/runtime-src/api/routes.ts +29 -1338
- package/runtime-src/auth/index.ts +33 -296
- package/runtime-src/capture/index.ts +123 -466
- package/runtime-src/cli.ts +374 -1288
- package/runtime-src/client/index.ts +30 -696
- package/runtime-src/execution/index.ts +143 -694
- package/runtime-src/graph/index.ts +10 -128
- package/runtime-src/intent-match.ts +27 -27
- package/runtime-src/kuri/client.ts +242 -1210
- package/runtime-src/orchestrator/index.ts +499 -2753
- package/runtime-src/reverse-engineer/index.ts +22 -239
- package/runtime-src/runtime/local-server.ts +22 -238
- package/runtime-src/runtime/paths.ts +5 -9
- package/runtime-src/runtime/setup.ts +1 -56
- package/runtime-src/server.ts +11 -11
- package/runtime-src/transform/schema-hints.ts +358 -0
- package/runtime-src/types/skill.ts +2 -488
- package/runtime-src/verification/index.ts +14 -35
- package/runtime-src/version.ts +8 -68
- package/vendor/kuri/darwin-arm64/kuri +0 -0
- package/vendor/kuri/darwin-x64/kuri +0 -0
- package/vendor/kuri/linux-arm64/kuri +0 -0
- package/vendor/kuri/linux-x64/kuri +0 -0
- package/SKILL.md +0 -754
- package/bin/unbrowse-update-hint.mjs +0 -22
- package/bin/unbrowse-wrapper.mjs +0 -107
- package/bin/unbrowse.js +0 -37
- package/dist/mcp.js +0 -1796
- package/runtime-src/agent-outcome.ts +0 -166
- package/runtime-src/analytics-session.ts +0 -55
- package/runtime-src/api/browse-index.ts +0 -254
- package/runtime-src/api/browse-session.ts +0 -648
- package/runtime-src/api/browse-submit-prereqs.ts +0 -48
- package/runtime-src/api/browse-submit.ts +0 -1184
- package/runtime-src/auth/runtime.ts +0 -116
- package/runtime-src/browser/index.ts +0 -643
- package/runtime-src/browser/types.ts +0 -41
- package/runtime-src/build-info.generated.ts +0 -4
- package/runtime-src/capture/prefetch.ts +0 -122
- package/runtime-src/capture/rsc.ts +0 -45
- package/runtime-src/cli/shortcuts.ts +0 -273
- package/runtime-src/client/graph-client.ts +0 -99
- package/runtime-src/execution/robots.ts +0 -167
- package/runtime-src/execution/search-forms.ts +0 -188
- package/runtime-src/graph/planner.ts +0 -411
- package/runtime-src/graph/session.ts +0 -294
- package/runtime-src/graph/trace-store.ts +0 -136
- package/runtime-src/indexer/index.ts +0 -441
- package/runtime-src/mcp.ts +0 -1522
- package/runtime-src/orchestrator/browser-agent.ts +0 -374
- package/runtime-src/orchestrator/dag-advisor.ts +0 -59
- package/runtime-src/orchestrator/dag-feedback.ts +0 -256
- package/runtime-src/orchestrator/first-pass-action.ts +0 -403
- package/runtime-src/orchestrator/passive-publish.ts +0 -182
- package/runtime-src/orchestrator/timing-economics.ts +0 -80
- package/runtime-src/payments/cascade.ts +0 -137
- package/runtime-src/payments/index.ts +0 -268
- package/runtime-src/payments/wallet.ts +0 -98
- package/runtime-src/publish/sanitize.ts +0 -197
- package/runtime-src/publish-admission.ts +0 -279
- package/runtime-src/reverse-engineer/description-prompt.ts +0 -213
- package/runtime-src/router.ts +0 -17
- package/runtime-src/routing-telemetry.ts +0 -395
- package/runtime-src/runtime/browser-access.ts +0 -11
- package/runtime-src/runtime/browser-auth.ts +0 -12
- package/runtime-src/runtime/browser-host.ts +0 -48
- package/runtime-src/runtime/lifecycle.ts +0 -17
- package/runtime-src/runtime/supervisor.ts +0 -69
- package/runtime-src/runtime/update-hints.ts +0 -351
- package/runtime-src/settings.ts +0 -221
- package/runtime-src/single-binary.ts +0 -141
- package/runtime-src/site-policy.ts +0 -54
- package/runtime-src/stale-cleanup-runner.ts +0 -144
- package/runtime-src/stale-cleanup.ts +0 -133
- package/runtime-src/telemetry-attribution.ts +0 -120
- package/runtime-src/telemetry.ts +0 -253
- package/runtime-src/verification/auth-gate.ts +0 -8
- package/runtime-src/verification/candidates.ts +0 -27
- package/runtime-src/verification/matrix.ts +0 -30
- package/runtime-src/workflow/artifact.ts +0 -161
- package/runtime-src/workflow/compile.ts +0 -808
- package/runtime-src/workflow/publish.ts +0 -205
- package/runtime-src/workflow/runtime.ts +0 -213
- package/scripts/postinstall.mjs +0 -105
- package/scripts/release-assets.mjs +0 -24
- package/scripts/verify-release-assets.mjs +0 -39
- package/vendor/kuri/manifest.json +0 -24
package/README.md
CHANGED
|
@@ -2,85 +2,28 @@
|
|
|
2
2
|
|
|
3
3
|
This package installs the `unbrowse` CLI.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Turn any website into a reusable API interface for agents. Unbrowse captures network traffic, reverse-engineers the real endpoints underneath the UI, and stores what it learns in a shared marketplace so the next agent can reuse it instantly.
|
|
6
6
|
|
|
7
7
|
One agent learns a site once. Every later agent gets the fast path.
|
|
8
8
|
|
|
9
|
-
Unbrowse is a drop-in replacement for OpenClaw / `agent-browser` browser flows for agents: on the API-native path it is typically ~30x faster, ~90% cheaper, and turns repeated browser work into reusable route assets.
|
|
10
|
-
|
|
11
9
|
> Security note: capture and execution stay local by default. Credentials stay on your machine. Learned API contracts are published to the shared marketplace only after capture. See [SKILL.md](./SKILL.md) for the full agent-facing API reference and tool-policy guidance.
|
|
12
10
|
|
|
13
|
-
## MCP server
|
|
14
|
-
|
|
15
|
-
Unbrowse implements the Model Context Protocol over stdio. `unbrowse mcp` is the MCP server entrypoint.
|
|
16
|
-
|
|
17
|
-
- Protocol: JSON-RPC 2.0 MCP over stdio
|
|
18
|
-
- Handshake: `initialize`, `notifications/initialized`, `ping`
|
|
19
|
-
- Capability surface today: `tools/list` and `tools/call`
|
|
20
|
-
- Current MCP shape: tools only. No `resources/*` or `prompts/*` methods yet.
|
|
21
|
-
- Runtime model: the MCP server fronts the local Unbrowse runtime on `http://localhost:6969`; hosts talk standard MCP, and Unbrowse uses the local HTTP runtime behind the scenes.
|
|
22
|
-
|
|
23
|
-
Core MCP tools:
|
|
24
|
-
|
|
25
|
-
- Discovery: `unbrowse_health`, `unbrowse_search`, `unbrowse_resolve`, `unbrowse_execute`, `unbrowse_feedback`
|
|
26
|
-
- Auth/cache: `unbrowse_login`, `unbrowse_skills`, `unbrowse_skill`, `unbrowse_sessions`
|
|
27
|
-
- Browser capture: `unbrowse_go`, `unbrowse_snap`, `unbrowse_click`, `unbrowse_fill`, `unbrowse_type`, `unbrowse_press`, `unbrowse_select`, `unbrowse_scroll`, `unbrowse_submit`, `unbrowse_screenshot`, `unbrowse_text`, `unbrowse_markdown`, `unbrowse_cookies`, `unbrowse_eval`, `unbrowse_sync`, `unbrowse_close`
|
|
28
|
-
|
|
29
|
-
Typical MCP host config:
|
|
30
|
-
|
|
31
|
-
```json
|
|
32
|
-
{
|
|
33
|
-
"mcpServers": {
|
|
34
|
-
"unbrowse": {
|
|
35
|
-
"command": "npx",
|
|
36
|
-
"args": ["-y", "unbrowse", "mcp"]
|
|
37
|
-
}
|
|
38
|
-
}
|
|
39
|
-
}
|
|
40
|
-
```
|
|
41
|
-
|
|
42
11
|
## Quick start
|
|
43
12
|
|
|
44
13
|
```bash
|
|
45
|
-
#
|
|
46
|
-
|
|
14
|
+
# Fastest full setup
|
|
15
|
+
npx unbrowse setup
|
|
47
16
|
```
|
|
48
17
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
```bash
|
|
52
|
-
# Deterministic setup from a repo clone
|
|
53
|
-
git clone --single-branch --depth 1 https://github.com/unbrowse-ai/unbrowse.git ~/unbrowse
|
|
54
|
-
cd ~/unbrowse && ./setup --host off
|
|
55
|
-
```
|
|
18
|
+
`npx unbrowse setup` downloads the CLI on demand, verifies the bundled Kuri runtime, lets you register with an email-shaped display identity, registers the Open Code `/unbrowse` command when Open Code is detected, and starts the local server.
|
|
56
19
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
If a wallet is configured, that wallet address becomes the contributor/payment truth: it is synced onto your agent profile, used as the destination for contributor payouts when your routes earn, and used as the spending wallet for paid marketplace routes.
|
|
60
|
-
|
|
61
|
-
Recommended for new installs: set up Crossmint `lobster.cash` during bootstrap. `unbrowse setup` now encourages it, and when the tooling is already present it will try `npx @crossmint/lobster-cli setup` automatically. That wallet becomes the payout destination for contributed routes and the spending wallet for paid marketplace routes.
|
|
62
|
-
|
|
63
|
-
Unbrowse supports wallet providers such as Crossmint `lobster.cash` for x402-gated routes. If you use `lobster.cash`, set `LOBSTER_WALLET_ADDRESS`. Other providers can use `AGENT_WALLET_ADDRESS` and optional `AGENT_WALLET_PROVIDER`.
|
|
64
|
-
|
|
65
|
-
For repeat npm use after a healthy publish:
|
|
20
|
+
For daily use:
|
|
66
21
|
|
|
67
22
|
```bash
|
|
68
23
|
npm install -g unbrowse
|
|
69
24
|
unbrowse setup
|
|
70
25
|
```
|
|
71
26
|
|
|
72
|
-
The npm package is binary-first: install downloads the prebuilt Bun-compiled CLI for your platform. It does not ship the TypeScript runtime or JS fallback launcher.
|
|
73
|
-
On supported platforms, install now fails fast if the matching GitHub release asset is missing or unreachable.
|
|
74
|
-
|
|
75
|
-
For generic MCP hosts:
|
|
76
|
-
|
|
77
|
-
```bash
|
|
78
|
-
git clone --single-branch --depth 1 https://github.com/unbrowse-ai/unbrowse.git ~/unbrowse
|
|
79
|
-
cd ~/unbrowse && ./setup --host mcp
|
|
80
|
-
```
|
|
81
|
-
|
|
82
|
-
That writes a ready-to-import MCP config to `~/.config/unbrowse/mcp/unbrowse.json`. A generic template is also published at [`/mcp.json`](https://www.unbrowse.ai/mcp.json).
|
|
83
|
-
|
|
84
27
|
If your agent host uses skills:
|
|
85
28
|
|
|
86
29
|
```bash
|
|
@@ -91,28 +34,11 @@ npx skills add unbrowse-ai/unbrowse
|
|
|
91
34
|
|
|
92
35
|
Unbrowse no longer self-updates at runtime. If you already have Unbrowse installed, upgrade to the latest version after each release or the new flow may not work on your machine.
|
|
93
36
|
|
|
94
|
-
|
|
37
|
+
If you installed the CLI globally:
|
|
95
38
|
|
|
96
39
|
```bash
|
|
97
|
-
unbrowse
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
Codex and Claude installs now also get a session-start update hint during `unbrowse setup`, so newer releases are surfaced in the host before the CLI drifts too far behind.
|
|
101
|
-
|
|
102
|
-
If you installed from a repo clone:
|
|
103
|
-
|
|
104
|
-
```bash
|
|
105
|
-
cd ~/unbrowse
|
|
106
|
-
git pull --ff-only
|
|
107
|
-
./setup --host off
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
If you installed for a generic MCP host:
|
|
111
|
-
|
|
112
|
-
```bash
|
|
113
|
-
cd ~/unbrowse
|
|
114
|
-
git pull --ff-only
|
|
115
|
-
./setup --host mcp
|
|
40
|
+
npm install -g unbrowse@latest
|
|
41
|
+
unbrowse setup
|
|
116
42
|
```
|
|
117
43
|
|
|
118
44
|
If your agent host uses skills, rerun its skill install/update command too:
|
|
@@ -123,44 +49,27 @@ npx skills add unbrowse-ai/unbrowse
|
|
|
123
49
|
|
|
124
50
|
Need help or want release updates? Join the Discord: [discord.gg/VWugEeFNsG](https://discord.gg/VWugEeFNsG)
|
|
125
51
|
|
|
126
|
-
Every CLI command auto-starts the local
|
|
127
|
-
Public companion docs: [docs.unbrowse.ai](https://docs.unbrowse.ai)
|
|
52
|
+
Every CLI command auto-starts the local server on `http://localhost:6969` by default. Override with `UNBROWSE_URL`, `PORT`, or `HOST`. On first startup it auto-registers as an agent with the marketplace and caches credentials in `~/.unbrowse/config.json`. `unbrowse setup` now prompts for an email-shaped identity first; headless setups can provide `UNBROWSE_AGENT_EMAIL`.
|
|
128
53
|
|
|
129
54
|
Works with Claude Code, Open Code, Cursor, Codex, Windsurf, and any agent host that can call a local CLI or skill.
|
|
130
55
|
|
|
131
56
|
## What setup does
|
|
132
57
|
|
|
133
|
-
- Checks
|
|
134
|
-
- Prebuilds the packaged CLI runtime and installs the stable `unbrowse` shim for the repo bootstrap path.
|
|
58
|
+
- Checks local prerequisites for the npm/npx flow.
|
|
135
59
|
- Verifies the bundled Kuri binary, or builds it from the vendored Kuri source when working from repo source with Zig installed.
|
|
136
60
|
- Registers the Open Code `/unbrowse` command when Open Code is present.
|
|
137
|
-
- Runs the first-use flow: ToS, agent registration/API-key caching, wallet detection, and Crossmint `lobster.cash` encouragement.
|
|
138
61
|
- Starts the local Unbrowse server unless `--no-start` is passed.
|
|
139
62
|
|
|
140
63
|
## Common commands
|
|
141
64
|
|
|
142
65
|
```bash
|
|
143
66
|
unbrowse health
|
|
144
|
-
unbrowse mcp
|
|
145
67
|
unbrowse resolve --intent "get trending searches" --url "https://google.com" --pretty
|
|
146
68
|
unbrowse login --url "https://calendar.google.com"
|
|
147
69
|
unbrowse skills
|
|
148
70
|
unbrowse search --intent "get stock prices"
|
|
149
71
|
```
|
|
150
72
|
|
|
151
|
-
For most MCP hosts, the standard flow is `unbrowse_resolve` first, then `unbrowse_execute`. For JS-heavy or first-time capture workflows, use the browser tool chain: `unbrowse_go -> unbrowse_snap -> action tools -> unbrowse_submit/unbrowse_sync -> unbrowse_close`.
|
|
152
|
-
|
|
153
|
-
## Dependency walk for multi-step UIs
|
|
154
|
-
|
|
155
|
-
Treat each successful browser submit as a dependency boundary.
|
|
156
|
-
|
|
157
|
-
- Do not jump straight to guessed downstream URLs like `/date-selection.html` or `/payment.html` unless the current session already reached them through the real page flow.
|
|
158
|
-
- Use `unbrowse_submit` for the actual transition, then trust the returned `url`, `session_id`, and any next-step hints over your own assumptions.
|
|
159
|
-
- `unbrowse_sync` after a good transition so the route graph records which request chain unlocked the next page.
|
|
160
|
-
- If a page later returns `abandonedCart`, `session_expired`, or a wrong audience/product variant, restart from the last known good upstream step and walk forward again.
|
|
161
|
-
|
|
162
|
-
The dependency graph is not just API-to-API. On JS-heavy checkout flows it also captures browser-state prerequisites: selected product, resident/non-resident audience, date, slot, auth, and cart state. Future agents should reason from those prerequisites before calling deeper steps.
|
|
163
|
-
|
|
164
73
|
## Demo notes
|
|
165
74
|
|
|
166
75
|
- First-time capture/indexing on a site can take 20-80 seconds. That is the slow path; repeats should be much faster.
|
|
@@ -171,21 +80,6 @@ The dependency graph is not just API-to-API. On JS-heavy checkout flows it also
|
|
|
171
80
|
|
|
172
81
|
If you tried Unbrowse on a site or API and could not get it to work, add it to [Discussion #53](https://github.com/unbrowse-ai/unbrowse/discussions/53). We use that thread to collect missing or broken targets so we can turn them into requirements for the next eval pass.
|
|
173
82
|
|
|
174
|
-
## Docs
|
|
175
|
-
|
|
176
|
-
The synced skill repo also carries the public docs set:
|
|
177
|
-
|
|
178
|
-
- [Quickstart](./docs/guides/quickstart.md)
|
|
179
|
-
- [API reference](./docs/api.md)
|
|
180
|
-
- [Deployment guide](./docs/deployment.md)
|
|
181
|
-
- [Release checklist](./docs/RELEASING.md)
|
|
182
|
-
|
|
183
|
-
Whitepaper companion docs:
|
|
184
|
-
|
|
185
|
-
- [Whitepaper companion index](./docs/whitepaper/README.md)
|
|
186
|
-
- [For Technical Readers](./docs/whitepaper/for-technical-readers.md)
|
|
187
|
-
- [For Investors](./docs/whitepaper/for-investors.md)
|
|
188
|
-
|
|
189
83
|
## How it works
|
|
190
84
|
|
|
191
85
|
When an agent asks for something, Unbrowse first searches the marketplace for an existing skill. If one exists with enough confidence, it executes immediately. If not, Unbrowse captures the site, learns the APIs behind it, publishes a reusable skill, and executes that instead.
|
|
@@ -241,7 +135,6 @@ Non-GET endpoints (POST, PUT, DELETE) require explicit confirmation:
|
|
|
241
135
|
|
|
242
136
|
- `dry_run: true` — preview what would execute without side effects
|
|
243
137
|
- `confirm_unsafe: true` — explicit user consent to proceed
|
|
244
|
-
- `confirm_third_party_terms: true` — extra explicit confirmation for policy-sensitive domains/actions such as X write endpoints
|
|
245
138
|
|
|
246
139
|
GET endpoints auto-execute. Mutations never fire without opt-in.
|
|
247
140
|
|
|
@@ -264,16 +157,6 @@ See [SKILL.md](./SKILL.md) for the full API reference including all endpoints, s
|
|
|
264
157
|
| GET | `/v1/stats/summary` | Platform stats |
|
|
265
158
|
| GET | `/health` | Health check |
|
|
266
159
|
|
|
267
|
-
## Docs
|
|
268
|
-
|
|
269
|
-
The standalone skill repo also carries the core repo docs:
|
|
270
|
-
|
|
271
|
-
- [Quickstart guide](./docs/guides/quickstart.md)
|
|
272
|
-
- [API notes](./docs/api.md)
|
|
273
|
-
- [Codex eval harness](./docs/codex-eval-harness.md)
|
|
274
|
-
- [Deployment notes](./docs/deployment.md)
|
|
275
|
-
- [Release checklist](./docs/RELEASING.md)
|
|
276
|
-
|
|
277
160
|
## Configuration
|
|
278
161
|
|
|
279
162
|
### Runtime directories
|