unbrowse 12.2.1 → 12.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SKILL.md +9 -1
- package/dist/cli.js +473 -45
- package/package.json +1 -1
- package/references/tools.json +59 -0
package/SKILL.md
CHANGED
|
@@ -63,7 +63,7 @@ Hosts that need a stdio server or reject dotted tool names can run `npx unbrowse
|
|
|
63
63
|
- CLI: `unbrowse discover`, `run`, `inspect`, `resume`, `scrape <url>` (one page as markdown), `index <url>` (teach a site; `index status`), `registry`, `site`, `whoami`. `scrape` and `index` need CLI 12.2.0 or later. `unbrowse run` never drives a website in a browser (the only browser it opens is the first-use sign-in). There is no `unbrowse browse` and no `unbrowse install`. `unbrowse help` describes the installed version.
|
|
64
64
|
- SDK: REST integration and scripting. Consult the public SDK docs for its supported methods; MCP tools and REST methods are not interchangeable names.
|
|
65
65
|
|
|
66
|
-
Core MCP tools: `unbrowse_discover`, `unbrowse_run`, `unbrowse_inspect`, `unbrowse_resume`, `unbrowse_cancel`, `unbrowse_scrape`, `unbrowse_map`, `unbrowse_sites`, `unbrowse_usage`, `unbrowse_credits`, `unbrowse_forget`, `unbrowse_learn`, `unbrowse_index`, `unbrowse_index_status`, `unbrowse_credentials_list`, `unbrowse_credentials_request`, `unbrowse_credentials_status`, `unbrowse_connect` (connect an app's own MCP server, such as Linear, Notion or GitHub, so its tools run as the person: an app with sign-in returns a link the person opens), and the cloud browser `unbrowse_browse_open`, `unbrowse_browse_snapshot`, `unbrowse_browse_act`, `unbrowse_browse_finish`, `unbrowse_browse_close`.
|
|
66
|
+
Core MCP tools: `unbrowse_discover`, `unbrowse_run`, `unbrowse_inspect`, `unbrowse_resume`, `unbrowse_cancel`, `unbrowse_scrape`, `unbrowse_map`, `unbrowse_sites`, `unbrowse_usage`, `unbrowse_credits`, `unbrowse_forget`, `unbrowse_learn`, `unbrowse_index`, `unbrowse_index_status`, `unbrowse_credentials_list`, `unbrowse_credentials_request`, `unbrowse_credentials_status`, `unbrowse_connect` (connect an app's own MCP server, such as Linear, Notion or GitHub, so its tools run as the person: an app with sign-in returns a link the person opens), `unbrowse_import_mcps` (bring the MCP servers this agent already has into Unbrowse, below), and the cloud browser `unbrowse_browse_open`, `unbrowse_browse_snapshot`, `unbrowse_browse_act`, `unbrowse_browse_finish`, `unbrowse_browse_close`.
|
|
67
67
|
|
|
68
68
|
[references/tools.json](references/tools.json) contains the exported core MCP input schemas. The connected server's `tools/list` is authoritative: it also includes dynamic tools available to this user's workspace. Never invent a capability ID or input schema.
|
|
69
69
|
|
|
@@ -112,6 +112,14 @@ To cover a whole site ahead of need, `unbrowse_index {url, focus?, maxCapabiliti
|
|
|
112
112
|
|
|
113
113
|
An existing HAR pair can be sent through `unbrowse_learn`. Only submit recordings the user authorized; HARs can contain private data. Private and loopback destinations are refused by the hosted service.
|
|
114
114
|
|
|
115
|
+
## Bring the agent's MCP servers
|
|
116
|
+
|
|
117
|
+
When the person wants their existing MCP servers in Unbrowse (one sign-in, every agent gets them), import them instead of re-adding each by hand.
|
|
118
|
+
|
|
119
|
+
- From a terminal: `npx unbrowse mcps` lists the servers in this machine's agent configs (Claude Code, Claude Desktop, Cursor, Codex, VS Code, Windsurf, Gemini, Zed); `npx unbrowse mcps import` imports them (`--dry-run` shows the plan). Keys in those configs stay on the machine unless the person adds `--with-keys`.
|
|
120
|
+
- As an MCP tool: read your host's MCP config yourself and call `unbrowse_import_mcps {servers:[{name, url} | {name, command, args}]}`. Never pass keys, headers or env values; `dryRun: true` shows the plan.
|
|
121
|
+
- Remote servers connect by address. A local (stdio) server cannot run in Unbrowse's cloud: when the catalog has the same app hosted, that is connected instead; otherwise it is `local_only` and stays in the agent. `sign_in` and `key` rows carry the link the person opens. Show them the plan before importing.
|
|
122
|
+
|
|
115
123
|
## Website sign-in
|
|
116
124
|
|
|
117
125
|
Unbrowse account sign-in and a website's saved login are separate. Never ask for passwords in chat or type credentials via ordinary tool arguments.
|
package/dist/cli.js
CHANGED
|
@@ -25,6 +25,9 @@ var __esm = (fn, res, err) => () => {
|
|
|
25
25
|
return res;
|
|
26
26
|
};
|
|
27
27
|
|
|
28
|
+
// ../sdk/src/version.ts
|
|
29
|
+
var SDK_VERSION = "12.3.0";
|
|
30
|
+
|
|
28
31
|
// ../sdk/src/client.ts
|
|
29
32
|
function apiBase(url) {
|
|
30
33
|
const u = url.replace(/\/+$/, "");
|
|
@@ -35,10 +38,12 @@ class Unbrowse {
|
|
|
35
38
|
baseUrl;
|
|
36
39
|
apiKey;
|
|
37
40
|
fetchImpl;
|
|
41
|
+
client;
|
|
38
42
|
constructor(opts = {}) {
|
|
39
43
|
this.baseUrl = apiBase(opts.baseUrl ?? env("UNBROWSE_BASE_URL") ?? DEFAULT_BASE_URL);
|
|
40
44
|
this.apiKey = opts.apiKey ?? env("UNBROWSE_API_KEY");
|
|
41
45
|
this.fetchImpl = opts.fetch ?? ((...a) => globalThis.fetch(...a));
|
|
46
|
+
this.client = (opts.client ?? `sdk/${SDK_VERSION}`).replace(/[^A-Za-z0-9._/-]/g, "").slice(0, 64);
|
|
42
47
|
}
|
|
43
48
|
async send(path, init) {
|
|
44
49
|
const res = await this.fetchImpl(`${this.baseUrl}${path}`, {
|
|
@@ -46,6 +51,7 @@ class Unbrowse {
|
|
|
46
51
|
headers: {
|
|
47
52
|
...this.apiKey ? { authorization: `Bearer ${this.apiKey}` } : {},
|
|
48
53
|
"content-type": "application/json",
|
|
54
|
+
...this.client ? { "x-unbrowse-client": this.client } : {},
|
|
49
55
|
...init?.headers ?? {}
|
|
50
56
|
}
|
|
51
57
|
});
|
|
@@ -158,7 +164,15 @@ class Unbrowse {
|
|
|
158
164
|
if (finished)
|
|
159
165
|
return;
|
|
160
166
|
try {
|
|
161
|
-
|
|
167
|
+
let next;
|
|
168
|
+
try {
|
|
169
|
+
next = await this.answerEgress(egressId, req.id, answer);
|
|
170
|
+
} catch (err) {
|
|
171
|
+
if (!(err instanceof UnbrowseError && err.status === 413) || !("response" in answer))
|
|
172
|
+
throw err;
|
|
173
|
+
const bytes = Math.round((answer.response.body?.length ?? 0) * 0.75);
|
|
174
|
+
next = await this.answerEgress(egressId, req.id, { error: `the site's response (${bytes} bytes) is too large to relay from this machine; run it from Unbrowse instead (CLI: --from-unbrowse)` });
|
|
175
|
+
}
|
|
162
176
|
if (isEgressStep(next) || isEgressWaiting(next))
|
|
163
177
|
latest = next;
|
|
164
178
|
else
|
|
@@ -232,6 +246,13 @@ class Unbrowse {
|
|
|
232
246
|
connect: (a) => this.post("/accounts/connections", a),
|
|
233
247
|
register: (a) => this.post("/accounts/register", a)
|
|
234
248
|
};
|
|
249
|
+
connectors = {
|
|
250
|
+
list: () => this.req("/connectors"),
|
|
251
|
+
apps: (q = "", limit = 20) => this.req(`/connectors/apps?q=${encodeURIComponent(q)}&limit=${limit}`),
|
|
252
|
+
connect: (c) => this.post("/connectors/connect", c),
|
|
253
|
+
import: (servers, opts = {}) => this.post("/connectors/import", { servers, ...opts.dryRun ? { dryRun: true } : {} }),
|
|
254
|
+
disconnect: (app) => this.post("/connectors/disconnect", { app })
|
|
255
|
+
};
|
|
235
256
|
vault() {
|
|
236
257
|
return this.req("/vault");
|
|
237
258
|
}
|
|
@@ -421,17 +442,208 @@ var init_src = __esm(() => {
|
|
|
421
442
|
init_mcp();
|
|
422
443
|
});
|
|
423
444
|
|
|
445
|
+
// src/agent-mcps.ts
|
|
446
|
+
import { existsSync as existsSync2, readFileSync as readFileSync2 } from "node:fs";
|
|
447
|
+
import { homedir as homedir2 } from "node:os";
|
|
448
|
+
import { join as join2 } from "node:path";
|
|
449
|
+
function parseJsonc(text) {
|
|
450
|
+
let out = "";
|
|
451
|
+
for (let i = 0;i < text.length; i++) {
|
|
452
|
+
const c = text[i];
|
|
453
|
+
if (c === '"') {
|
|
454
|
+
let j = i + 1;
|
|
455
|
+
while (j < text.length && text[j] !== '"')
|
|
456
|
+
j += text[j] === "\\" ? 2 : 1;
|
|
457
|
+
out += text.slice(i, j + 1);
|
|
458
|
+
i = j;
|
|
459
|
+
} else if (c === "/" && text[i + 1] === "/") {
|
|
460
|
+
while (i < text.length && text[i] !== `
|
|
461
|
+
`)
|
|
462
|
+
i++;
|
|
463
|
+
out += `
|
|
464
|
+
`;
|
|
465
|
+
} else if (c === "/" && text[i + 1] === "*") {
|
|
466
|
+
i = text.indexOf("*/", i + 2);
|
|
467
|
+
if (i < 0)
|
|
468
|
+
break;
|
|
469
|
+
i++;
|
|
470
|
+
} else
|
|
471
|
+
out += c;
|
|
472
|
+
}
|
|
473
|
+
return JSON.parse(out.replace(/,(\s*[}\]])/g, "$1"));
|
|
474
|
+
}
|
|
475
|
+
function parseCodexToml(text) {
|
|
476
|
+
const servers = {};
|
|
477
|
+
let current;
|
|
478
|
+
let sub;
|
|
479
|
+
const value = (v) => {
|
|
480
|
+
v = v.trim();
|
|
481
|
+
if (/^".*"$/.test(v) || /^'.*'$/.test(v))
|
|
482
|
+
return v.slice(1, -1).replace(/\\"/g, '"');
|
|
483
|
+
if (v.startsWith("["))
|
|
484
|
+
return [...v.matchAll(/"((?:[^"\\]|\\.)*)"|'([^']*)'/g)].map((m) => m[1] ?? m[2]);
|
|
485
|
+
if (v.startsWith("{"))
|
|
486
|
+
return Object.fromEntries([...v.matchAll(/([\w.-]+|"[^"]+")\s*=\s*("(?:[^"\\]|\\.)*"|'[^']*')/g)].map((m) => [m[1].replace(/"/g, ""), m[2].slice(1, -1)]));
|
|
487
|
+
if (v === "true" || v === "false")
|
|
488
|
+
return v === "true";
|
|
489
|
+
return v;
|
|
490
|
+
};
|
|
491
|
+
for (const line of text.split(/\r?\n/)) {
|
|
492
|
+
const t = line.replace(/\s+#.*$/, "").trim();
|
|
493
|
+
if (!t || t.startsWith("#"))
|
|
494
|
+
continue;
|
|
495
|
+
const table = /^\[mcp_servers\.("?)([^\]".]+)\1(?:\.([\w-]+))?\]$/.exec(t);
|
|
496
|
+
if (table) {
|
|
497
|
+
current = servers[table[2]] ??= {};
|
|
498
|
+
sub = table[3];
|
|
499
|
+
if (sub)
|
|
500
|
+
current[sub] ??= {};
|
|
501
|
+
continue;
|
|
502
|
+
}
|
|
503
|
+
if (t.startsWith("[")) {
|
|
504
|
+
current = undefined;
|
|
505
|
+
continue;
|
|
506
|
+
}
|
|
507
|
+
const kv = /^([\w.-]+|"[^"]+")\s*=\s*(.+)$/.exec(t);
|
|
508
|
+
if (!kv || !current)
|
|
509
|
+
continue;
|
|
510
|
+
const key = kv[1].replace(/"/g, "");
|
|
511
|
+
if (sub)
|
|
512
|
+
current[sub][key] = value(kv[2]);
|
|
513
|
+
else
|
|
514
|
+
current[key] = value(kv[2]);
|
|
515
|
+
}
|
|
516
|
+
return servers;
|
|
517
|
+
}
|
|
518
|
+
function places(home = process.env.HOME || process.env.USERPROFILE || homedir2(), cwd = process.cwd(), platform = process.platform, env = process.env) {
|
|
519
|
+
const appData = env.APPDATA ?? join2(home, "AppData", "Roaming");
|
|
520
|
+
const desktop = platform === "darwin" ? join2(home, "Library", "Application Support", "Claude") : platform === "win32" ? join2(appData, "Claude") : join2(env.XDG_CONFIG_HOME ?? join2(home, ".config"), "Claude");
|
|
521
|
+
const code = platform === "darwin" ? join2(home, "Library", "Application Support", "Code", "User") : platform === "win32" ? join2(appData, "Code", "User") : join2(env.XDG_CONFIG_HOME ?? join2(home, ".config"), "Code", "User");
|
|
522
|
+
const zed = platform === "darwin" || platform === "linux" ? join2(env.XDG_CONFIG_HOME ?? join2(home, ".config"), "zed", "settings.json") : join2(appData, "Zed", "settings.json");
|
|
523
|
+
const mcpServers = (t) => asObj(parseJsonc(t).mcpServers);
|
|
524
|
+
return [
|
|
525
|
+
{ host: "claude-code", file: join2(home, ".claude.json"), read: (t) => {
|
|
526
|
+
const j = parseJsonc(t);
|
|
527
|
+
const projects = asObj(j.projects);
|
|
528
|
+
return { ...asObj(projects[cwd]?.mcpServers), ...asObj(j.mcpServers) };
|
|
529
|
+
} },
|
|
530
|
+
{ host: "claude-code", file: join2(cwd, ".mcp.json"), read: mcpServers },
|
|
531
|
+
{ host: "claude-desktop", file: join2(desktop, "claude_desktop_config.json"), read: mcpServers },
|
|
532
|
+
{ host: "cursor", file: join2(home, ".cursor", "mcp.json"), read: mcpServers },
|
|
533
|
+
{ host: "cursor", file: join2(cwd, ".cursor", "mcp.json"), read: mcpServers },
|
|
534
|
+
{ host: "codex", file: join2(env.CODEX_HOME ?? join2(home, ".codex"), "config.toml"), read: parseCodexToml },
|
|
535
|
+
{ host: "vscode", file: join2(code, "mcp.json"), read: (t) => asObj(parseJsonc(t).servers) },
|
|
536
|
+
{ host: "vscode", file: join2(cwd, ".vscode", "mcp.json"), read: (t) => asObj(parseJsonc(t).servers) },
|
|
537
|
+
{ host: "windsurf", file: join2(home, ".codeium", "windsurf", "mcp_config.json"), read: mcpServers },
|
|
538
|
+
{ host: "gemini", file: join2(home, ".gemini", "settings.json"), read: mcpServers },
|
|
539
|
+
{ host: "zed", file: zed, read: (t) => asObj(parseJsonc(t).context_servers) }
|
|
540
|
+
];
|
|
541
|
+
}
|
|
542
|
+
function normalize(host, file, name, raw) {
|
|
543
|
+
const zedCommand = asObj(raw.command);
|
|
544
|
+
const command = str(raw.command) ?? str(zedCommand.path);
|
|
545
|
+
const args = (Array.isArray(raw.args) ? raw.args : Array.isArray(zedCommand.args) ? zedCommand.args : []).filter((a) => typeof a === "string").map((a) => secretish(a) && !/^https:\/\/[^\s?#=]+$/.test(a) ? HIDDEN : a);
|
|
546
|
+
const url = str(raw.url) ?? str(raw.serverUrl) ?? str(raw.httpUrl);
|
|
547
|
+
const type = String(raw.type ?? raw.transport ?? "").toLowerCase();
|
|
548
|
+
const transport = url ? type === "sse" || !type && /\/sse\/?$/.test(url) ? "sse" : "streamable-http" : undefined;
|
|
549
|
+
const headers = { ...asObj(raw.headers), ...asObj(raw.http_headers) };
|
|
550
|
+
const env = { ...asObj(raw.env), ...asObj(zedCommand.env) };
|
|
551
|
+
const self = !!url && isUnbrowse(url) || /unbrowse/.test(`${command ?? ""} ${args.join(" ")}`) && args.includes("mcp");
|
|
552
|
+
return { host, file, name, ...url ? { url, transport } : {}, ...command ? { command, args } : {}, headerNames: Object.keys(headers), envNames: Object.keys(env), self };
|
|
553
|
+
}
|
|
554
|
+
function scan(opts = {}) {
|
|
555
|
+
const servers = [];
|
|
556
|
+
const skipped = [];
|
|
557
|
+
const seenFiles = new Set;
|
|
558
|
+
for (const p of places(opts.home, opts.cwd, opts.platform, opts.env)) {
|
|
559
|
+
if (seenFiles.has(p.file) || !existsSync2(p.file))
|
|
560
|
+
continue;
|
|
561
|
+
seenFiles.add(p.file);
|
|
562
|
+
try {
|
|
563
|
+
for (const [name, raw] of Object.entries(p.read(readFileSync2(p.file, "utf8")))) {
|
|
564
|
+
if (raw && typeof raw === "object" && raw.disabled !== true && raw.enabled !== false)
|
|
565
|
+
servers.push(normalize(p.host, p.file, name, raw));
|
|
566
|
+
}
|
|
567
|
+
} catch (e) {
|
|
568
|
+
skipped.push({ file: p.file, why: e.message.slice(0, 120) });
|
|
569
|
+
}
|
|
570
|
+
}
|
|
571
|
+
return { servers, skipped };
|
|
572
|
+
}
|
|
573
|
+
function toImport(f) {
|
|
574
|
+
if (f.url)
|
|
575
|
+
return { name: f.name, url: f.url, ...f.transport ? { transport: f.transport } : {} };
|
|
576
|
+
const args = (f.args ?? []).filter((a) => a !== HIDDEN && (/^https:\/\/[^\s?#]+$/.test(a) || !a.startsWith("-") && !secretish(a) && /^[@\w][\w@./:-]*$/.test(a) && /mcp|server|remote|gateway/i.test(a)));
|
|
577
|
+
return { name: f.name, ...f.command ? { command: f.command.split(/[\\/]/).pop() } : {}, ...args.length ? { args } : {} };
|
|
578
|
+
}
|
|
579
|
+
function keyOf(f, raw) {
|
|
580
|
+
if (!f.url)
|
|
581
|
+
return;
|
|
582
|
+
const headers = { ...asObj(raw.headers), ...asObj(raw.http_headers) };
|
|
583
|
+
for (const [h, v] of Object.entries(headers)) {
|
|
584
|
+
if (typeof v !== "string" || !v.trim() || /\$\{|\$[A-Z_]/.test(v))
|
|
585
|
+
continue;
|
|
586
|
+
if (/^(authorization|x-api-key|api-key|x-auth-token)$/i.test(h))
|
|
587
|
+
return { header: h.toLowerCase(), key: v.trim() };
|
|
588
|
+
}
|
|
589
|
+
return;
|
|
590
|
+
}
|
|
591
|
+
function rawEntry(f, opts = {}) {
|
|
592
|
+
const p = places(opts.home, opts.cwd, opts.platform, opts.env).find((x) => x.file === f.file && x.host === f.host);
|
|
593
|
+
try {
|
|
594
|
+
return p ? p.read(readFileSync2(f.file, "utf8"))[f.name] : undefined;
|
|
595
|
+
} catch {
|
|
596
|
+
return;
|
|
597
|
+
}
|
|
598
|
+
}
|
|
599
|
+
function unique(servers) {
|
|
600
|
+
const seen = new Set;
|
|
601
|
+
return servers.filter((s) => {
|
|
602
|
+
const k = s.url ? `u:${s.url.replace(/\/+$/, "")}` : `c:${s.command} ${(s.args ?? []).join(" ")}`;
|
|
603
|
+
if (seen.has(k))
|
|
604
|
+
return false;
|
|
605
|
+
seen.add(k);
|
|
606
|
+
return true;
|
|
607
|
+
});
|
|
608
|
+
}
|
|
609
|
+
var asObj = (v) => v && typeof v === "object" && !Array.isArray(v) ? v : {}, str = (v) => typeof v === "string" && v.trim() ? v.trim() : undefined, secretish = (a) => /[=]/.test(a) || /(token|secret|key|password|bearer)/i.test(a) || /[A-Za-z0-9_\-]{32,}/.test(a) || /^(sk|ghp|gho|xox[abp]|ub_live|AKIA)[-_A-Za-z0-9]/.test(a), HIDDEN = "‹hidden›", isUnbrowse = (url) => /^https?:\/\/((www|v3|stage|api)\.)?unbrowse\.ai(\/|$)/i.test(url) || /^https?:\/\/[a-z0-9-]+-mcp\.unbrowse\.ai(\/|$)/i.test(url);
|
|
610
|
+
var init_agent_mcps = () => {};
|
|
611
|
+
|
|
424
612
|
// src/mcp-proxy.ts
|
|
425
613
|
import { createInterface } from "node:readline";
|
|
426
614
|
function safeToolName(name) {
|
|
427
615
|
return name.replace(/[^A-Za-z0-9_-]/g, "_").slice(0, 64);
|
|
428
616
|
}
|
|
429
|
-
function
|
|
617
|
+
async function localMcps(args, upstream) {
|
|
618
|
+
const A = await Promise.resolve().then(() => (init_agent_mcps(), {}));
|
|
619
|
+
const { servers, skipped } = scan();
|
|
620
|
+
const names = Array.isArray(args.names) ? new Set(args.names.map(String)) : undefined;
|
|
621
|
+
const pick = servers.filter((s) => !names || names.has(s.name));
|
|
622
|
+
const listing = pick.map(({ file: _f, ...s }) => s);
|
|
623
|
+
if (args.import !== true) {
|
|
624
|
+
return { content: [{ type: "text", text: JSON.stringify({ servers: listing, skipped: skipped.map((x) => x.why) }, null, 2) }], structuredContent: { servers: listing }, isError: false };
|
|
625
|
+
}
|
|
626
|
+
const send = unique(pick.filter((s) => !s.self)).map(toImport);
|
|
627
|
+
if (!send.length)
|
|
628
|
+
return { content: [{ type: "text", text: "No MCP servers to import." }], isError: false };
|
|
629
|
+
return (await upstream()).rpc("tools/call", { name: "unbrowse_import_mcps", arguments: { servers: send, ...args.dryRun === true ? { dryRun: true } : {} } });
|
|
630
|
+
}
|
|
631
|
+
function localRun(rest, log = (l) => void process.stderr.write(`${l}
|
|
632
|
+
`)) {
|
|
633
|
+
return async (args) => {
|
|
634
|
+
const request = Object.fromEntries(RUN_KEYS.filter((k) => args[k] !== undefined).map((k) => [k, args[k]]));
|
|
635
|
+
const deadline = Number(args.deadlineMs);
|
|
636
|
+
const view = await (await rest()).runOnClient(request, { onRequest: (r) => log(`→ ${r.method} ${r.url}`), timeoutMs: Math.max(120000, Number.isFinite(deadline) ? deadline : 0) });
|
|
637
|
+
const text = JSON.stringify(view);
|
|
638
|
+
return { content: [{ type: "text", text: text.length > RESULT_MAX ? `${text.slice(0, RESULT_MAX)}…` : text }], structuredContent: view, isError: false };
|
|
639
|
+
};
|
|
640
|
+
}
|
|
641
|
+
function createProxy(upstream, version, opts = {}) {
|
|
430
642
|
const names = new Map;
|
|
431
643
|
async function list() {
|
|
432
644
|
const tools = await (await upstream()).listTools();
|
|
433
645
|
names.clear();
|
|
434
|
-
const out = [];
|
|
646
|
+
const out = [LOCAL_MCPS];
|
|
435
647
|
for (const t of tools) {
|
|
436
648
|
const safe = safeToolName(t.name);
|
|
437
649
|
if (names.has(safe))
|
|
@@ -467,6 +679,10 @@ function createProxy(upstream, version) {
|
|
|
467
679
|
return reply({ tools: await list() });
|
|
468
680
|
case "tools/call": {
|
|
469
681
|
const name = String(msg.params?.name ?? "");
|
|
682
|
+
if (name === LOCAL_MCPS.name)
|
|
683
|
+
return reply(await localMcps(msg.params?.arguments ?? {}, upstream));
|
|
684
|
+
if (opts.runLocal && await original(name) === "unbrowse_run")
|
|
685
|
+
return reply(await opts.runLocal(msg.params?.arguments ?? {}));
|
|
470
686
|
return reply(await (await upstream()).rpc("tools/call", { ...msg.params, name: await original(name) }));
|
|
471
687
|
}
|
|
472
688
|
default:
|
|
@@ -481,7 +697,7 @@ function createProxy(upstream, version) {
|
|
|
481
697
|
};
|
|
482
698
|
}
|
|
483
699
|
async function serveStdio(opts) {
|
|
484
|
-
const handle = createProxy(async () => new UnbrowseMcp({ url: opts.url, apiKey: await opts.token() ?? "", endUser: opts.endUser, client: `cli-mcp/${opts.version}` }), opts.version);
|
|
700
|
+
const handle = createProxy(async () => new UnbrowseMcp({ url: opts.url, apiKey: await opts.token() ?? "", endUser: opts.endUser, client: `cli-mcp/${opts.version}` }), opts.version, opts.fromHere ? { runLocal: localRun(opts.fromHere) } : {});
|
|
485
701
|
const write = (m) => new Promise((resolve) => process.stdout.write(`${JSON.stringify(m)}
|
|
486
702
|
`, () => resolve()));
|
|
487
703
|
const rl = createInterface({ input: process.stdin, crlfDelay: Infinity });
|
|
@@ -502,10 +718,24 @@ async function serveStdio(opts) {
|
|
|
502
718
|
}
|
|
503
719
|
await Promise.all(pending);
|
|
504
720
|
}
|
|
505
|
-
var PROTOCOLS;
|
|
721
|
+
var PROTOCOLS, LOCAL_MCPS, RUN_KEYS, RESULT_MAX = 40000;
|
|
506
722
|
var init_mcp_proxy = __esm(() => {
|
|
507
723
|
init_src();
|
|
508
724
|
PROTOCOLS = ["2025-11-25", "2025-06-18", "2025-03-26", "2024-11-05"];
|
|
725
|
+
LOCAL_MCPS = {
|
|
726
|
+
name: "unbrowse_local_mcps",
|
|
727
|
+
description: "The MCP servers configured in this machine's agents (Claude Code, Claude Desktop, Cursor, Codex, VS Code, Windsurf, Gemini, Zed), read by this local Unbrowse server: name, agent, remote address or local command, and the NAMES of any headers and env vars (never their values). import: true brings them into Unbrowse through unbrowse_import_mcps (remote ones by address, local ones as their hosted app; Unbrowse itself and duplicates skipped); dryRun: true shows that plan. Keys stay on this machine; for a server that needs one, the person opens the returned link, or runs `unbrowse mcps import --with-keys` in a terminal.",
|
|
728
|
+
inputSchema: {
|
|
729
|
+
type: "object",
|
|
730
|
+
properties: {
|
|
731
|
+
import: { type: "boolean", description: "Import them into Unbrowse" },
|
|
732
|
+
dryRun: { type: "boolean", description: "With import: show the plan without connecting anything" },
|
|
733
|
+
names: { type: "array", items: { type: "string" }, description: "Only these servers (by their config name)" }
|
|
734
|
+
},
|
|
735
|
+
additionalProperties: false
|
|
736
|
+
}
|
|
737
|
+
};
|
|
738
|
+
RUN_KEYS = ["capability", "task", "targetUrl", "input", "accountRefs", "interactionMode", "idempotencyKey", "allowedDestinations"];
|
|
509
739
|
});
|
|
510
740
|
|
|
511
741
|
// src/cookie-daemon.ts
|
|
@@ -535,9 +765,9 @@ __export(exports_cookie_daemon, {
|
|
|
535
765
|
writeJson: () => writeJson
|
|
536
766
|
});
|
|
537
767
|
import { createHash as createHash2 } from "node:crypto";
|
|
538
|
-
import { existsSync as
|
|
539
|
-
import { homedir as
|
|
540
|
-
import { dirname as dirname2, join as
|
|
768
|
+
import { existsSync as existsSync3, mkdirSync as mkdirSync2, readFileSync as readFileSync3, writeFileSync as writeFileSync2 } from "node:fs";
|
|
769
|
+
import { homedir as homedir3, platform } from "node:os";
|
|
770
|
+
import { dirname as dirname2, join as join3 } from "node:path";
|
|
541
771
|
function siteOf(domain) {
|
|
542
772
|
const labels = domain.replace(/^\./, "").toLowerCase().split(".");
|
|
543
773
|
if (labels.length <= 2)
|
|
@@ -550,7 +780,7 @@ function configDir(cliConfigPath) {
|
|
|
550
780
|
}
|
|
551
781
|
function readJson(path, fallback) {
|
|
552
782
|
try {
|
|
553
|
-
return
|
|
783
|
+
return existsSync3(path) ? JSON.parse(readFileSync3(path, "utf8")) : fallback;
|
|
554
784
|
} catch {
|
|
555
785
|
return fallback;
|
|
556
786
|
}
|
|
@@ -694,10 +924,10 @@ async function watch(opts) {
|
|
|
694
924
|
}
|
|
695
925
|
function servicePlan(opts) {
|
|
696
926
|
const os = opts.os ?? platform();
|
|
697
|
-
const home = opts.home ??
|
|
927
|
+
const home = opts.home ?? homedir3();
|
|
698
928
|
const env = Object.entries(opts.env ?? {});
|
|
699
929
|
if (os === "linux") {
|
|
700
|
-
const path =
|
|
930
|
+
const path = join3(home, ".config", "systemd", "user", `${SERVICE}.service`);
|
|
701
931
|
const content = [
|
|
702
932
|
"[Unit]",
|
|
703
933
|
"Description=Unbrowse cookie sync: keeps your browser sign-ins for chosen sites in sync with Unbrowse",
|
|
@@ -725,8 +955,8 @@ function servicePlan(opts) {
|
|
|
725
955
|
};
|
|
726
956
|
}
|
|
727
957
|
if (os === "darwin") {
|
|
728
|
-
const path =
|
|
729
|
-
const logPath =
|
|
958
|
+
const path = join3(home, "Library", "LaunchAgents", `${LAUNCHD_LABEL}.plist`);
|
|
959
|
+
const logPath = join3(home, "Library", "Logs", `${SERVICE}.log`);
|
|
730
960
|
const esc = (s) => s.replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">");
|
|
731
961
|
const content = `<?xml version="1.0" encoding="UTF-8"?>
|
|
732
962
|
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
|
@@ -764,7 +994,7 @@ function quoteSystemd(s) {
|
|
|
764
994
|
function ephemeralCli(cli) {
|
|
765
995
|
return /[\\/](_npx|\.npm[\\/]_npx|npx-[^\\/]+|tmp|Temp)[\\/]/i.test(cli);
|
|
766
996
|
}
|
|
767
|
-
var ALL_SITES = "*", allSites = (cfg) => cfg.sites.includes(ALL_SITES), UPLOAD_BATCH = 500, TWO_LEVEL, DEFAULT_INTERVAL_MIN = 15, REFRESH_MS, SERVICE = "unbrowse-cookies", LAUNCHD_LABEL = "ai.unbrowse.cookies", configFile = (dir) =>
|
|
997
|
+
var ALL_SITES = "*", allSites = (cfg) => cfg.sites.includes(ALL_SITES), UPLOAD_BATCH = 500, TWO_LEVEL, DEFAULT_INTERVAL_MIN = 15, REFRESH_MS, SERVICE = "unbrowse-cookies", LAUNCHD_LABEL = "ai.unbrowse.cookies", configFile = (dir) => join3(dir, "cookie-sync.json"), stateFile = (dir) => join3(dir, "cookie-sync.state.json");
|
|
768
998
|
var init_cookie_daemon = __esm(() => {
|
|
769
999
|
TWO_LEVEL = /^(co|com|net|org|gov|edu|ac|or|ne|go)\.[a-z]{2}$/;
|
|
770
1000
|
REFRESH_MS = 24 * 60 * 60000;
|
|
@@ -988,18 +1218,18 @@ var HEADER = 100;
|
|
|
988
1218
|
// src/cookies.ts
|
|
989
1219
|
import { createDecipheriv, createHash as createHash3, pbkdf2Sync } from "node:crypto";
|
|
990
1220
|
import { execFileSync } from "node:child_process";
|
|
991
|
-
import { existsSync as
|
|
992
|
-
import { homedir as
|
|
993
|
-
import { join as
|
|
1221
|
+
import { existsSync as existsSync4, readdirSync, readFileSync as readFileSync4 } from "node:fs";
|
|
1222
|
+
import { homedir as homedir4, platform as platform2 } from "node:os";
|
|
1223
|
+
import { join as join4 } from "node:path";
|
|
994
1224
|
function chromium(name, [mac, linux], keychain) {
|
|
995
1225
|
return {
|
|
996
1226
|
name,
|
|
997
1227
|
engine: "chromium",
|
|
998
1228
|
keychain: `${keychain} Safe Storage`,
|
|
999
1229
|
dirs: (h) => ({
|
|
1000
|
-
darwin: [
|
|
1001
|
-
linux: [
|
|
1002
|
-
win32: [
|
|
1230
|
+
darwin: [join4(h, "Library/Application Support", mac)],
|
|
1231
|
+
linux: [join4(h, ".config", linux), join4(h, ".var/app", flatpakId(name), "config", linux)],
|
|
1232
|
+
win32: [join4(process.env.LOCALAPPDATA ?? join4(h, "AppData/Local"), mac.replace(/\//g, "\\"), "User Data")]
|
|
1003
1233
|
})
|
|
1004
1234
|
};
|
|
1005
1235
|
}
|
|
@@ -1008,9 +1238,9 @@ function firefox(name, [mac, linux, linuxHome]) {
|
|
|
1008
1238
|
name,
|
|
1009
1239
|
engine: "firefox",
|
|
1010
1240
|
dirs: (h) => ({
|
|
1011
|
-
darwin: [
|
|
1012
|
-
linux: [
|
|
1013
|
-
win32: [
|
|
1241
|
+
darwin: [join4(h, "Library/Application Support", mac)],
|
|
1242
|
+
linux: [join4(h, ".config", linux), join4(h, linuxHome ?? `.${name.toLowerCase()}`)],
|
|
1243
|
+
win32: [join4(process.env.APPDATA ?? join4(h, "AppData/Roaming"), mac)]
|
|
1014
1244
|
})
|
|
1015
1245
|
};
|
|
1016
1246
|
}
|
|
@@ -1022,7 +1252,7 @@ function findProfiles() {
|
|
|
1022
1252
|
const raw = [];
|
|
1023
1253
|
for (const def of BROWSERS) {
|
|
1024
1254
|
for (const base of def.dirs(home())[OS] ?? []) {
|
|
1025
|
-
if (!
|
|
1255
|
+
if (!existsSync4(base))
|
|
1026
1256
|
continue;
|
|
1027
1257
|
raw.push(...def.engine === "chromium" ? chromiumProfiles(def, base) : firefoxProfiles(def, base));
|
|
1028
1258
|
}
|
|
@@ -1043,7 +1273,7 @@ function chromiumProfiles(def, base) {
|
|
|
1043
1273
|
const out = [];
|
|
1044
1274
|
let names = [];
|
|
1045
1275
|
try {
|
|
1046
|
-
const local = JSON.parse(
|
|
1276
|
+
const local = JSON.parse(readFileSync4(join4(base, "Local State"), "utf8"));
|
|
1047
1277
|
names = Object.keys(local.profile?.info_cache ?? {});
|
|
1048
1278
|
} catch {
|
|
1049
1279
|
names = ["Default"];
|
|
@@ -1051,8 +1281,8 @@ function chromiumProfiles(def, base) {
|
|
|
1051
1281
|
if (!names.length)
|
|
1052
1282
|
names = ["Default"];
|
|
1053
1283
|
for (const dir of names) {
|
|
1054
|
-
const path =
|
|
1055
|
-
if (
|
|
1284
|
+
const path = join4(base, dir);
|
|
1285
|
+
if (existsSync4(join4(path, "Cookies")) || existsSync4(join4(path, "Network", "Cookies"))) {
|
|
1056
1286
|
out.push({ browser: def.name, profile: dir, label: `${def.name} · ${dir}`, path, engine: "chromium" });
|
|
1057
1287
|
}
|
|
1058
1288
|
}
|
|
@@ -1062,14 +1292,14 @@ function firefoxProfiles(def, base) {
|
|
|
1062
1292
|
const out = [];
|
|
1063
1293
|
let dirs = [];
|
|
1064
1294
|
try {
|
|
1065
|
-
const ini =
|
|
1295
|
+
const ini = readFileSync4(join4(base, "profiles.ini"), "utf8");
|
|
1066
1296
|
dirs = [...ini.matchAll(/^Path=(.+)$/gm)].map((m) => m[1].trim());
|
|
1067
1297
|
} catch {
|
|
1068
1298
|
dirs = safeReaddir(base).filter((d) => /\.default|\.dev-edition/.test(d));
|
|
1069
1299
|
}
|
|
1070
1300
|
for (const rel of dirs) {
|
|
1071
|
-
const path =
|
|
1072
|
-
if (
|
|
1301
|
+
const path = join4(base, rel);
|
|
1302
|
+
if (existsSync4(join4(path, "cookies.sqlite")))
|
|
1073
1303
|
out.push({ browser: def.name, profile: rel, label: `${def.name} · ${rel}`, path, engine: "firefox" });
|
|
1074
1304
|
}
|
|
1075
1305
|
return out;
|
|
@@ -1085,7 +1315,7 @@ function readCookies(profile, opts = {}) {
|
|
|
1085
1315
|
return profile.engine === "chromium" ? chromiumCookies(profile, opts) : firefoxCookies(profile, opts);
|
|
1086
1316
|
}
|
|
1087
1317
|
function loadDb(path) {
|
|
1088
|
-
return new Uint8Array(
|
|
1318
|
+
return new Uint8Array(readFileSync4(path));
|
|
1089
1319
|
}
|
|
1090
1320
|
function domainMatch(host, want) {
|
|
1091
1321
|
if (!want)
|
|
@@ -1095,7 +1325,7 @@ function domainMatch(host, want) {
|
|
|
1095
1325
|
return h === w || h.endsWith(`.${w}`);
|
|
1096
1326
|
}
|
|
1097
1327
|
function chromiumCookies(profile, opts) {
|
|
1098
|
-
const dbPath =
|
|
1328
|
+
const dbPath = existsSync4(join4(profile.path, "Network", "Cookies")) ? join4(profile.path, "Network", "Cookies") : join4(profile.path, "Cookies");
|
|
1099
1329
|
const rows = readTable(loadDb(dbPath), "cookies");
|
|
1100
1330
|
const decryptor = chromiumDecryptor(profile.browser, rows);
|
|
1101
1331
|
const out = [];
|
|
@@ -1121,7 +1351,7 @@ function chromiumCookies(profile, opts) {
|
|
|
1121
1351
|
return out;
|
|
1122
1352
|
}
|
|
1123
1353
|
function firefoxCookies(profile, opts) {
|
|
1124
|
-
const rows = readTable(loadDb(
|
|
1354
|
+
const rows = readTable(loadDb(join4(profile.path, "cookies.sqlite")), "moz_cookies");
|
|
1125
1355
|
const out = [];
|
|
1126
1356
|
for (const r of rows) {
|
|
1127
1357
|
const host = String(r.host ?? "");
|
|
@@ -1262,7 +1492,7 @@ function keychainName(browser) {
|
|
|
1262
1492
|
function windowsKey(_browser) {
|
|
1263
1493
|
return null;
|
|
1264
1494
|
}
|
|
1265
|
-
var home = () => process.env.UNBROWSE_COOKIES_HOME ??
|
|
1495
|
+
var home = () => process.env.UNBROWSE_COOKIES_HOME ?? homedir4(), OS, BROWSERS, TEXT;
|
|
1266
1496
|
var init_cookies = __esm(() => {
|
|
1267
1497
|
OS = platform2();
|
|
1268
1498
|
BROWSERS = [
|
|
@@ -1283,7 +1513,7 @@ var init_cookies = __esm(() => {
|
|
|
1283
1513
|
// src/cli.ts
|
|
1284
1514
|
import { spawn, spawnSync } from "node:child_process";
|
|
1285
1515
|
import { randomUUID } from "node:crypto";
|
|
1286
|
-
import { existsSync as
|
|
1516
|
+
import { existsSync as existsSync5, mkdirSync as mkdirSync3, readFileSync as readFileSync5, realpathSync, rmSync as rmSync2, writeFileSync as writeFileSync3 } from "node:fs";
|
|
1287
1517
|
import { dirname as dirname3 } from "node:path";
|
|
1288
1518
|
import { createInterface as createInterface2 } from "node:readline/promises";
|
|
1289
1519
|
|
|
@@ -1326,6 +1556,34 @@ function v1Key(baseUrl) {
|
|
|
1326
1556
|
return;
|
|
1327
1557
|
}
|
|
1328
1558
|
}
|
|
1559
|
+
function sharedSignIn(baseUrl, now = Date.now()) {
|
|
1560
|
+
if (process.env.UNBROWSE_SHARED_SIGNIN === "0")
|
|
1561
|
+
return;
|
|
1562
|
+
const dir = process.env.CLAUDE_CONFIG_DIR ?? join(process.env.HOME || homedir(), ".claude");
|
|
1563
|
+
let entries;
|
|
1564
|
+
try {
|
|
1565
|
+
entries = Object.values(JSON.parse(readFileSync(join(dir, ".credentials.json"), "utf8")).mcpOAuth ?? {});
|
|
1566
|
+
} catch {
|
|
1567
|
+
return;
|
|
1568
|
+
}
|
|
1569
|
+
const api = new URL(apiResource(baseUrl));
|
|
1570
|
+
for (const e of entries) {
|
|
1571
|
+
if (typeof e?.serverUrl !== "string" || typeof e.accessToken !== "string" || !e.accessToken)
|
|
1572
|
+
continue;
|
|
1573
|
+
if (typeof e.expiresAt === "number" && now > e.expiresAt - 60000)
|
|
1574
|
+
continue;
|
|
1575
|
+
let url;
|
|
1576
|
+
try {
|
|
1577
|
+
url = new URL(e.serverUrl);
|
|
1578
|
+
} catch {
|
|
1579
|
+
continue;
|
|
1580
|
+
}
|
|
1581
|
+
if (url.origin !== api.origin || !(url.pathname === api.pathname || url.pathname.startsWith(`${api.pathname}/`)))
|
|
1582
|
+
continue;
|
|
1583
|
+
return { token: e.accessToken, from: `Claude Code's MCP sign-in (${String(e.serverName ?? url.origin + url.pathname)})` };
|
|
1584
|
+
}
|
|
1585
|
+
return;
|
|
1586
|
+
}
|
|
1329
1587
|
async function currentToken(baseUrl, fetchImpl = fetch) {
|
|
1330
1588
|
if (process.env.UNBROWSE_API_KEY)
|
|
1331
1589
|
return process.env.UNBROWSE_API_KEY;
|
|
@@ -1336,7 +1594,7 @@ async function currentToken(baseUrl, fetchImpl = fetch) {
|
|
|
1336
1594
|
return stored.apiKey;
|
|
1337
1595
|
const o = stored.oauth;
|
|
1338
1596
|
if (!o)
|
|
1339
|
-
return v1Key(baseUrl);
|
|
1597
|
+
return v1Key(baseUrl) ?? sharedSignIn(baseUrl)?.token;
|
|
1340
1598
|
if (o.expiresAt && o.refreshToken && Date.now() > o.expiresAt - 60000) {
|
|
1341
1599
|
try {
|
|
1342
1600
|
const fresh = await tokenRequest(baseUrl, { grant_type: "refresh_token", refresh_token: o.refreshToken, client_id: o.clientId, resource: apiResource(baseUrl) }, fetchImpl);
|
|
@@ -1348,6 +1606,18 @@ async function currentToken(baseUrl, fetchImpl = fetch) {
|
|
|
1348
1606
|
}
|
|
1349
1607
|
return o.accessToken;
|
|
1350
1608
|
}
|
|
1609
|
+
function signInSource(baseUrl) {
|
|
1610
|
+
if (process.env.UNBROWSE_API_KEY)
|
|
1611
|
+
return "env";
|
|
1612
|
+
const stored = load();
|
|
1613
|
+
if (stored.baseUrl && stored.baseUrl !== baseUrl)
|
|
1614
|
+
return;
|
|
1615
|
+
if (stored.apiKey || stored.oauth)
|
|
1616
|
+
return "stored";
|
|
1617
|
+
if (v1Key(baseUrl))
|
|
1618
|
+
return "v1";
|
|
1619
|
+
return sharedSignIn(baseUrl) ? "shared" : undefined;
|
|
1620
|
+
}
|
|
1351
1621
|
function describe(baseUrl) {
|
|
1352
1622
|
if (process.env.UNBROWSE_API_KEY)
|
|
1353
1623
|
return "API key from UNBROWSE_API_KEY";
|
|
@@ -1361,6 +1631,9 @@ function describe(baseUrl) {
|
|
|
1361
1631
|
const v1 = !stored.baseUrl ? v1Key(baseUrl) : undefined;
|
|
1362
1632
|
if (v1)
|
|
1363
1633
|
return `v1 API key ${mask(v1)} from ~/.unbrowse/config.json (still works; \`unbrowse login\` replaces it)`;
|
|
1634
|
+
const shared = !stored.baseUrl ? sharedSignIn(baseUrl) : undefined;
|
|
1635
|
+
if (shared)
|
|
1636
|
+
return `${shared.from}, shared (\`unbrowse login\` gives the CLI its own)`;
|
|
1364
1637
|
return "not signed in";
|
|
1365
1638
|
}
|
|
1366
1639
|
function mask(key) {
|
|
@@ -1445,7 +1718,7 @@ ${url.toString()}`);
|
|
|
1445
1718
|
|
|
1446
1719
|
// src/cli.ts
|
|
1447
1720
|
init_src();
|
|
1448
|
-
var VERSION = "12.
|
|
1721
|
+
var VERSION = "12.3.0";
|
|
1449
1722
|
var DEFAULT_ORIGIN = "https://unbrowse.ai";
|
|
1450
1723
|
var HELP = `unbrowse ${VERSION} — call websites as APIs through the Unbrowse API
|
|
1451
1724
|
|
|
@@ -1477,6 +1750,10 @@ var HELP = `unbrowse ${VERSION} — call websites as APIs through the Unbrowse A
|
|
|
1477
1750
|
cookies daemon start|status|stop
|
|
1478
1751
|
The same as a background service (systemd / launchd) [--domain a,b | --all-sites] [--yes]
|
|
1479
1752
|
|
|
1753
|
+
mcps MCP servers in this machine's agents (Claude Code, Cursor, Codex, VS Code, …)
|
|
1754
|
+
mcps import [name…] Bring them into Unbrowse: remote ones by address, local ones as their hosted app
|
|
1755
|
+
[--dry-run] [--with-keys] keys in those configs stay here unless --with-keys (asks first; --yes in scripts)
|
|
1756
|
+
|
|
1480
1757
|
learn <a.har> <b.har>… Compile two HAR recordings into a capability [--title T] [--goal G]
|
|
1481
1758
|
learned [id] Your learned capabilities, or one with its harness and skill
|
|
1482
1759
|
logins Saved logins as masked hints
|
|
@@ -1490,11 +1767,15 @@ var HELP = `unbrowse ${VERSION} — call websites as APIs through the Unbrowse A
|
|
|
1490
1767
|
|
|
1491
1768
|
mcp Local stdio MCP server proxying the hosted one (tool names use _ not .)
|
|
1492
1769
|
[--url URL] [--end-user ID] for agent hosts that need stdio or strict tool names (Grok Build)
|
|
1770
|
+
[--from-here] unbrowse_run sends its site requests from this machine, like run (a Grok Bot's computer)
|
|
1771
|
+
mcp add [claude] [codex] Register that proxy with Claude Code / Codex: they use this CLI's sign-in (no second sign-in)
|
|
1772
|
+
[--name NAME] [--replace] [--dry-run]
|
|
1773
|
+
And back: no CLI sign-in, but Claude Code signed in to Unbrowse (plugin) → the CLI uses that
|
|
1493
1774
|
|
|
1494
1775
|
Options: --json (errors as JSON) · --base-url URL · --no-open
|
|
1495
1776
|
Env: UNBROWSE_API_KEY, UNBROWSE_BASE_URL (default ${DEFAULT_ORIGIN}), UNBROWSE_EGRESS=server, UNBROWSE_MCP_URL, UNBROWSE_END_USER
|
|
1496
1777
|
Exit: 0 ok · 1 error · 2 input required · 3 sign-in or login needed · 4 not verified`;
|
|
1497
|
-
var BOOLEAN = new Set(["json", "help", "version", "no-open", "no-wait", "unattended", "from-here", "from-unbrowse", "all", "all-sites", "yes", "full", "events"]);
|
|
1778
|
+
var BOOLEAN = new Set(["json", "help", "version", "no-open", "no-wait", "unattended", "from-here", "from-unbrowse", "all", "all-sites", "yes", "full", "events", "dry-run", "with-keys", "replace"]);
|
|
1498
1779
|
var NEEDS_ACCOUNT = new Set(["whoami", "usage", "discover", "run", "scrape", "index", "inspect", "resume", "cancel", "learn", "learned", "logins", "call", "cookies"]);
|
|
1499
1780
|
var SCRAPE_FORMATS = ["markdown", "html", "text", "links", "raw"];
|
|
1500
1781
|
function parseArgs(argv) {
|
|
@@ -1556,14 +1837,15 @@ async function main(argv, io = stdio) {
|
|
|
1556
1837
|
const origin = String(args.flags["base-url"] ?? process.env.UNBROWSE_BASE_URL ?? load().baseUrl ?? DEFAULT_ORIGIN).replace(/\/+$/, "");
|
|
1557
1838
|
const flag = (k) => typeof args.flags[k] === "string" ? args.flags[k] : undefined;
|
|
1558
1839
|
const print = (v) => io.out(JSON.stringify(v, null, 2));
|
|
1559
|
-
const client = async () => new Unbrowse({ apiKey: await currentToken(origin) ?? "", baseUrl: origin });
|
|
1560
|
-
if (cmd === "mcp") {
|
|
1840
|
+
const client = async () => new Unbrowse({ apiKey: await currentToken(origin) ?? "", baseUrl: origin, client: `cli/${VERSION}` });
|
|
1841
|
+
if (cmd === "mcp" && rest[0] !== "add") {
|
|
1561
1842
|
await Promise.resolve().then(() => init_mcp_proxy());
|
|
1562
1843
|
await serveStdio({
|
|
1563
1844
|
url: flag("url") ?? process.env.UNBROWSE_MCP_URL ?? `${origin}/api/mcp`,
|
|
1564
1845
|
token: async () => await currentToken(origin) ?? undefined,
|
|
1565
1846
|
version: VERSION,
|
|
1566
|
-
endUser: flag("end-user") ?? process.env.UNBROWSE_END_USER
|
|
1847
|
+
endUser: flag("end-user") ?? process.env.UNBROWSE_END_USER,
|
|
1848
|
+
...args.flags["from-here"] || process.env.UNBROWSE_EGRESS === "client" ? { fromHere: client } : {}
|
|
1567
1849
|
});
|
|
1568
1850
|
return 0;
|
|
1569
1851
|
}
|
|
@@ -1639,7 +1921,7 @@ async function main(argv, io = stdio) {
|
|
|
1639
1921
|
if (!io.interactive || !io.ask || args.flags.json)
|
|
1640
1922
|
return;
|
|
1641
1923
|
const { D, dir } = await cookieSync();
|
|
1642
|
-
if (
|
|
1924
|
+
if (existsSync5(D.configFile(dir)))
|
|
1643
1925
|
return;
|
|
1644
1926
|
io.err("Optional: let runs act as you on sites you are signed into in your browser. Unbrowse keeps those sites' cookies in sync in the background (sealed in your vault; only the sites you name leave this machine).");
|
|
1645
1927
|
const answer = (await io.ask("Sites to keep in sync (e.g. github.com, linkedin.com), or Enter to skip: ")).trim();
|
|
@@ -1681,6 +1963,8 @@ Your browser opens ${origin}; approve it and \`unbrowse ${cmd}\` carries on. (Sc
|
|
|
1681
1963
|
await offerCookieSync().catch((e) => io.err(`Cookie sync not set up: ${e.message}`));
|
|
1682
1964
|
return 0;
|
|
1683
1965
|
}
|
|
1966
|
+
case "mcp":
|
|
1967
|
+
return await mcpAdd(rest.slice(1), args, io, origin);
|
|
1684
1968
|
case "logout":
|
|
1685
1969
|
clear();
|
|
1686
1970
|
io.err("Signed out.");
|
|
@@ -1832,7 +2116,7 @@ Your browser opens ${origin}; approve it and \`unbrowse ${cmd}\` carries on. (Sc
|
|
|
1832
2116
|
if (plan)
|
|
1833
2117
|
for (const cmd of plan.disable)
|
|
1834
2118
|
exec(cmd);
|
|
1835
|
-
if (plan &&
|
|
2119
|
+
if (plan && existsSync5(plan.path))
|
|
1836
2120
|
rmSync2(plan.path, { force: true });
|
|
1837
2121
|
rmSync2(D.configFile(dir), { force: true });
|
|
1838
2122
|
rmSync2(D.stateFile(dir), { force: true });
|
|
@@ -1841,7 +2125,7 @@ Your browser opens ${origin}; approve it and \`unbrowse ${cmd}\` carries on. (Sc
|
|
|
1841
2125
|
}
|
|
1842
2126
|
if (action === "status") {
|
|
1843
2127
|
const state = D.readJson(D.stateFile(dir), { sites: {} });
|
|
1844
|
-
const installed = !!plan &&
|
|
2128
|
+
const installed = !!plan && existsSync5(plan.path);
|
|
1845
2129
|
const active = installed && plan ? exec(plan.status).status === 0 : false;
|
|
1846
2130
|
const out = {
|
|
1847
2131
|
installed,
|
|
@@ -1921,13 +2205,15 @@ Your browser opens ${origin}; approve it and \`unbrowse ${cmd}\` carries on. (Sc
|
|
|
1921
2205
|
}
|
|
1922
2206
|
case "learn": {
|
|
1923
2207
|
need(rest[0], "learn <a.har> <b.har>…");
|
|
1924
|
-
const har = rest.map((f) => JSON.parse(
|
|
2208
|
+
const har = rest.map((f) => JSON.parse(readFileSync5(f, "utf8")));
|
|
1925
2209
|
print(await ub.learn({ har: har.length === 1 ? har[0] : har, ...flag("title") ? { title: flag("title") } : {}, ...flag("goal") ? { goal: flag("goal") } : {} }));
|
|
1926
2210
|
return 0;
|
|
1927
2211
|
}
|
|
1928
2212
|
case "learned":
|
|
1929
2213
|
print(await ub.learned(rest[0]));
|
|
1930
2214
|
return 0;
|
|
2215
|
+
case "mcps":
|
|
2216
|
+
return await mcps(rest, args, io, origin, print, client, onboard);
|
|
1931
2217
|
case "logins":
|
|
1932
2218
|
if (rest[0] === "remove")
|
|
1933
2219
|
print(await ub.logins.remove({ origin: need(rest[1], "logins remove <origin>") }));
|
|
@@ -2021,6 +2307,148 @@ async function settle(ub, view, args, io, print) {
|
|
|
2021
2307
|
io.err(`${view.status}: ${view.error.code}${view.error.message ? ` — ${view.error.message}` : ""}`);
|
|
2022
2308
|
return 1;
|
|
2023
2309
|
}
|
|
2310
|
+
async function mcpAdd(rest, args, io, origin) {
|
|
2311
|
+
const HOSTS = ["claude", "codex"];
|
|
2312
|
+
const unknown = rest.filter((h) => !HOSTS.includes(h));
|
|
2313
|
+
if (unknown.length)
|
|
2314
|
+
throw new UsageError(`usage: unbrowse mcp add [claude] [codex] [--name NAME] [--replace] [--dry-run] (unknown: ${unknown.join(", ")})`);
|
|
2315
|
+
const name = (typeof args.flags.name === "string" ? args.flags.name : "") || "unbrowse";
|
|
2316
|
+
const exec = io.exec ?? (() => ({ status: 1, stdout: "", stderr: "no exec" }));
|
|
2317
|
+
const dry = !!args.flags["dry-run"];
|
|
2318
|
+
const replace = !!args.flags.replace;
|
|
2319
|
+
const win = (io.service?.os ?? process.platform) === "win32";
|
|
2320
|
+
const shell = (cmd) => win ? ["cmd", "/c", ...cmd] : cmd;
|
|
2321
|
+
await Promise.resolve().then(() => init_cookie_daemon());
|
|
2322
|
+
let cli = io.service?.cli ?? process.argv[1] ?? "";
|
|
2323
|
+
try {
|
|
2324
|
+
cli = realpathSync(cli);
|
|
2325
|
+
} catch {}
|
|
2326
|
+
const launch = [...io.service?.cli || !ephemeralCli(cli) ? [io.service?.node ?? process.execPath, cli] : shell(["npx", "-y", "unbrowse"]), "mcp", ...origin !== DEFAULT_ORIGIN ? ["--base-url", origin] : []];
|
|
2327
|
+
const env = ["UNBROWSE_CONFIG_DIR", "XDG_CONFIG_HOME", "CLAUDE_CONFIG_DIR", "UNBROWSE_SHARED_SIGNIN"].flatMap((k) => process.env[k] ? [`${k}=${process.env[k]}`] : []);
|
|
2328
|
+
const plans = {
|
|
2329
|
+
claude: { get: ["claude", "mcp", "get", name], remove: ["claude", "mcp", "remove", name, "--scope", "user"], add: ["claude", "mcp", "add", "--scope", "user", name, ...env.flatMap((e) => ["-e", e]), "--", ...launch] },
|
|
2330
|
+
codex: { get: ["codex", "mcp", "get", name], remove: ["codex", "mcp", "remove", name], add: ["codex", "mcp", "add", name, ...env.flatMap((e) => ["--env", e]), "--", ...launch] }
|
|
2331
|
+
};
|
|
2332
|
+
const wanted = rest.length ? rest : HOSTS.filter((h) => dry || exec(shell([h, "--version"])).status === 0);
|
|
2333
|
+
if (!wanted.length)
|
|
2334
|
+
return io.err("No Claude Code or Codex found on PATH. Any MCP host can run the proxy itself: " + launch.join(" ")), 1;
|
|
2335
|
+
let failed = 0;
|
|
2336
|
+
for (const h of wanted) {
|
|
2337
|
+
const plan = plans[h];
|
|
2338
|
+
if (dry) {
|
|
2339
|
+
io.out([...replace ? [plan.remove] : [], plan.add].map((c) => shell(c).join(" ")).join(`
|
|
2340
|
+
`));
|
|
2341
|
+
continue;
|
|
2342
|
+
}
|
|
2343
|
+
if (exec(shell(plan.get)).status === 0) {
|
|
2344
|
+
if (!replace) {
|
|
2345
|
+
failed++;
|
|
2346
|
+
io.err(`✗ ${h}: a server named "${name}" already exists; left as it is. Re-run with --replace, or pick --name.`);
|
|
2347
|
+
continue;
|
|
2348
|
+
}
|
|
2349
|
+
const r = exec(shell(plan.remove));
|
|
2350
|
+
if (r.status !== 0) {
|
|
2351
|
+
failed++;
|
|
2352
|
+
io.err(`✗ ${h}: could not remove the existing "${name}": ${(r.stderr || r.stdout).trim() || `exit ${r.status}`}`);
|
|
2353
|
+
continue;
|
|
2354
|
+
}
|
|
2355
|
+
}
|
|
2356
|
+
const r = exec(shell(plan.add));
|
|
2357
|
+
if (r.status === 0)
|
|
2358
|
+
io.err(`✓ ${h}: added "${name}" (stdio, on this CLI's sign-in)`);
|
|
2359
|
+
else
|
|
2360
|
+
failed++, io.err(`✗ ${h}: ${(r.stderr || r.stdout).trim() || `exit ${r.status}`}`);
|
|
2361
|
+
}
|
|
2362
|
+
if (!dry) {
|
|
2363
|
+
const source = signInSource(origin);
|
|
2364
|
+
if (source === "env")
|
|
2365
|
+
io.err("This CLI is signed in only through UNBROWSE_API_KEY, which the hosts do not get: run `unbrowse login` (or `unbrowse login --key`) so they have a sign-in.");
|
|
2366
|
+
else if (source === "shared")
|
|
2367
|
+
io.err("This CLI is using Claude Code's sign-in, which only Claude Code renews: run `unbrowse login` so the hosts keep working.");
|
|
2368
|
+
else if (!source)
|
|
2369
|
+
io.err("Not signed in yet: run `unbrowse login` once and the hosts use that sign-in.");
|
|
2370
|
+
}
|
|
2371
|
+
return failed ? 1 : 0;
|
|
2372
|
+
}
|
|
2373
|
+
async function mcps(rest, args, io, origin, print, client, onboard) {
|
|
2374
|
+
const A = await Promise.resolve().then(() => (init_agent_mcps(), {}));
|
|
2375
|
+
const { servers, skipped } = scan();
|
|
2376
|
+
const tilde = (f) => f.replace(process.env.HOME ?? "\x00", "~");
|
|
2377
|
+
if (rest[0] !== "import") {
|
|
2378
|
+
if (rest[0])
|
|
2379
|
+
throw new UsageError("usage: unbrowse mcps [import [name…] [--dry-run] [--with-keys]]");
|
|
2380
|
+
if (args.flags.json)
|
|
2381
|
+
return print({ servers: servers.map(({ file, ...s }) => ({ ...s, file: tilde(file) })), skipped }), 0;
|
|
2382
|
+
if (!servers.length)
|
|
2383
|
+
return io.err(`No MCP servers found in this machine's agent configs.${skipped.length ? ` Unreadable: ${skipped.map((x) => tilde(x.file)).join(", ")}` : ""}`), 0;
|
|
2384
|
+
let last = "";
|
|
2385
|
+
for (const s of servers) {
|
|
2386
|
+
const where = `${s.host} ${tilde(s.file)}`;
|
|
2387
|
+
if (where !== last)
|
|
2388
|
+
io.out(where), last = where;
|
|
2389
|
+
const what = s.url ? `remote ${s.url}` : `local ${[s.command, ...s.args ?? []].join(" ").slice(0, 90)}`;
|
|
2390
|
+
const env = s.envNames.length > 3 ? [...s.envNames.slice(0, 3).map((e) => `env ${e}`), `+${s.envNames.length - 3} env`] : s.envNames.map((e) => `env ${e}`);
|
|
2391
|
+
const secrets = [...s.headerNames.map((h) => `header ${h}`), ...env];
|
|
2392
|
+
io.out(` ${s.name.padEnd(22)} ${what}${secrets.length ? ` (${secrets.join(", ")})` : ""}${s.self ? " (Unbrowse itself)" : ""}`);
|
|
2393
|
+
}
|
|
2394
|
+
io.err(`
|
|
2395
|
+
${unique(servers.filter((s) => !s.self)).length} to import: unbrowse mcps import --dry-run`);
|
|
2396
|
+
return 0;
|
|
2397
|
+
}
|
|
2398
|
+
if (!await currentToken(origin) && !await onboard())
|
|
2399
|
+
return 3;
|
|
2400
|
+
const names = new Set(rest.slice(1));
|
|
2401
|
+
const pick = unique(servers.filter((s) => !s.self && (!names.size || names.has(s.name))));
|
|
2402
|
+
const missing = [...names].filter((n) => !servers.some((s) => s.name === n));
|
|
2403
|
+
if (missing.length)
|
|
2404
|
+
throw new UsageError(`no MCP server named ${missing.join(", ")} in this machine's agent configs (unbrowse mcps lists them)`);
|
|
2405
|
+
if (!pick.length)
|
|
2406
|
+
return io.err("Nothing to import."), 0;
|
|
2407
|
+
const dry = !!args.flags["dry-run"];
|
|
2408
|
+
const keyed = args.flags["with-keys"] ? pick.flatMap((f) => {
|
|
2409
|
+
const k = keyOf(f, rawEntry(f) ?? {});
|
|
2410
|
+
return k ? [{ f, k }] : [];
|
|
2411
|
+
}) : [];
|
|
2412
|
+
if (keyed.length && !dry && !args.flags.yes) {
|
|
2413
|
+
const list = keyed.map((x) => `${x.f.name} (${x.k.header})`).join(", ");
|
|
2414
|
+
if (!io.interactive || !io.ask)
|
|
2415
|
+
throw new UsageError(`Re-run with --yes to confirm: the keys for ${list} will be sent to ${origin} and sealed in your vault.`);
|
|
2416
|
+
const ok = (await io.ask(`Send the keys for ${list} from your agent configs to Unbrowse (sealed in your vault)? [y/N] `)).trim().toLowerCase();
|
|
2417
|
+
if (ok !== "y" && ok !== "yes")
|
|
2418
|
+
return io.err("Nothing sent."), 1;
|
|
2419
|
+
}
|
|
2420
|
+
const ub = await client();
|
|
2421
|
+
const rows = [];
|
|
2422
|
+
for (const { f, k } of keyed) {
|
|
2423
|
+
if (dry) {
|
|
2424
|
+
rows.push({ name: f.name, status: "planned", via: "url", note: `Would connect ${f.url} with the ${k.header} from its config.` });
|
|
2425
|
+
continue;
|
|
2426
|
+
}
|
|
2427
|
+
try {
|
|
2428
|
+
const r = await ub.connectors.connect({ url: f.url, name: f.name, ...f.transport ? { transport: f.transport } : {}, header: k.header, key: k.key });
|
|
2429
|
+
rows.push({ name: f.name, status: r.status, app: r.app, tools: r.tools, url: r.url, via: "url", note: r.status === "connected" ? `Connected with its key: ${r.tools} tools.` : `Needs the person: ${r.url}` });
|
|
2430
|
+
} catch (e) {
|
|
2431
|
+
rows.push({ name: f.name, status: "failed", note: e.message.slice(0, 200) });
|
|
2432
|
+
}
|
|
2433
|
+
}
|
|
2434
|
+
const rest2 = pick.filter((f) => !keyed.some((x) => x.f === f));
|
|
2435
|
+
if (rest2.length)
|
|
2436
|
+
rows.push(...(await ub.connectors.import(rest2.map(toImport), { dryRun: dry })).results);
|
|
2437
|
+
if (args.flags.json)
|
|
2438
|
+
print({ results: rows });
|
|
2439
|
+
else {
|
|
2440
|
+
const mark = { connected: "✓", planned: "·", sign_in: "→", key: "→", already: "=", skipped: "-", local_only: "-", failed: "✗" };
|
|
2441
|
+
for (const r of rows)
|
|
2442
|
+
io.out(`${mark[r.status] ?? "?"} ${r.name.padEnd(22)} ${r.status.padEnd(10)} ${r.note}${r.url ? `
|
|
2443
|
+
${r.url}` : ""}`);
|
|
2444
|
+
const hasKeyRows = rows.some((r) => r.status === "key") && !args.flags["with-keys"] && pick.some((f) => f.headerNames.length);
|
|
2445
|
+
if (hasKeyRows)
|
|
2446
|
+
io.err("A key server can take the key from your agent config instead: unbrowse mcps import --with-keys");
|
|
2447
|
+
if (dry)
|
|
2448
|
+
io.err("Nothing was connected. Run without --dry-run to import.");
|
|
2449
|
+
}
|
|
2450
|
+
return rows.some((r) => r.status === "failed") ? 1 : 0;
|
|
2451
|
+
}
|
|
2024
2452
|
function noCapabilityHint(view, args) {
|
|
2025
2453
|
const result = view.result ?? {};
|
|
2026
2454
|
const task = args._.slice(1).join(" ");
|
package/package.json
CHANGED
package/references/tools.json
CHANGED
|
@@ -549,6 +549,65 @@
|
|
|
549
549
|
"additionalProperties": false
|
|
550
550
|
}
|
|
551
551
|
},
|
|
552
|
+
{
|
|
553
|
+
"name": "unbrowse_import_mcps",
|
|
554
|
+
"description": "Import the MCP servers this agent already has into Unbrowse, so their tools run here with the person's own account and every agent signed in to Unbrowse gets them. Read your host's MCP config yourself (Claude Code: ~/.claude.json and .mcp.json; Claude Desktop: claude_desktop_config.json; Cursor: ~/.cursor/mcp.json; Codex: ~/.codex/config.toml; VS Code: .vscode/mcp.json; Windsurf: ~/.codeium/windsurf/mcp_config.json; Gemini: ~/.gemini/settings.json) and pass each server: a remote one as {name, url}, a local one as {name, command, args}. Never pass keys, headers or env values. A remote server is connected by its address (open ones at once; one with OAuth or a key returns the link the person opens). A local server cannot run in Unbrowse's cloud: when the catalog has the same app hosted, that is connected instead, otherwise it is reported local_only and stays in the agent. Servers already connected, duplicates and Unbrowse itself are skipped. dryRun: true shows the plan without connecting. CLI: `unbrowse mcps import` reads the configs itself.",
|
|
555
|
+
"inputSchema": {
|
|
556
|
+
"type": "object",
|
|
557
|
+
"properties": {
|
|
558
|
+
"servers": {
|
|
559
|
+
"type": "array",
|
|
560
|
+
"maxItems": 50,
|
|
561
|
+
"description": "The MCP servers in your agent's config",
|
|
562
|
+
"items": {
|
|
563
|
+
"type": "object",
|
|
564
|
+
"properties": {
|
|
565
|
+
"name": {
|
|
566
|
+
"type": "string",
|
|
567
|
+
"description": "The server's name in the config"
|
|
568
|
+
},
|
|
569
|
+
"url": {
|
|
570
|
+
"type": "string",
|
|
571
|
+
"description": "A remote server's https address"
|
|
572
|
+
},
|
|
573
|
+
"transport": {
|
|
574
|
+
"type": "string",
|
|
575
|
+
"enum": [
|
|
576
|
+
"streamable-http",
|
|
577
|
+
"http",
|
|
578
|
+
"sse"
|
|
579
|
+
],
|
|
580
|
+
"description": "A remote server's transport, when the config says"
|
|
581
|
+
},
|
|
582
|
+
"command": {
|
|
583
|
+
"type": "string",
|
|
584
|
+
"description": "A local server's command, e.g. npx"
|
|
585
|
+
},
|
|
586
|
+
"args": {
|
|
587
|
+
"type": "array",
|
|
588
|
+
"items": {
|
|
589
|
+
"type": "string"
|
|
590
|
+
},
|
|
591
|
+
"description": "A local server's arguments (package names; leave out tokens and paths to secrets)"
|
|
592
|
+
}
|
|
593
|
+
},
|
|
594
|
+
"required": [
|
|
595
|
+
"name"
|
|
596
|
+
],
|
|
597
|
+
"additionalProperties": false
|
|
598
|
+
}
|
|
599
|
+
},
|
|
600
|
+
"dryRun": {
|
|
601
|
+
"type": "boolean",
|
|
602
|
+
"description": "Show what would happen without connecting anything"
|
|
603
|
+
}
|
|
604
|
+
},
|
|
605
|
+
"required": [
|
|
606
|
+
"servers"
|
|
607
|
+
],
|
|
608
|
+
"additionalProperties": false
|
|
609
|
+
}
|
|
610
|
+
},
|
|
552
611
|
{
|
|
553
612
|
"name": "unbrowse_credits",
|
|
554
613
|
"description": "This workspace's credits: the free monthly allowance left (pays for Unbrowse's own verified calls only; resets monthly) and paid credits (bought by the user; never expire), plus recent credit history and the top-up link. 1 credit = one verified call = $0.001. Pass a pack (p10 = $10 → 10,000, p50 = $50 → 50,000, p200 = $200 → 200,000) to get a Stripe Checkout link for the user to open — you never pay or see card details.",
|