@rezkam/browser-tools 1.0.4 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +7 -0
- package/README.md +39 -1
- package/bin/browser-tools.mjs +2 -0
- package/package.json +4 -2
- package/scripts/browser-control.mjs +126 -17
- package/scripts/browser-platform.mjs +96 -0
- package/scripts/serve.mjs +42 -0
- package/scripts/tab-broker.mjs +351 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,12 @@
|
|
|
1
1
|
# @rezkam/browser-tools
|
|
2
2
|
|
|
3
|
+
## 1.1.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- 0fcffe1: Support native Linux Chrome/Chromium discovery, profile locations, headless diagnostics and platform-correct user agents while retaining Puppeteer and owner-protected lifecycle contracts. Verify fixed-port endpoints against launch arguments, read Linux process identity through procfs, and add Linux/macOS browser smoke CI.
|
|
8
|
+
- e7a80e3: Add `browser-tools serve`, a tab broker that runs one lean Chrome for every client on a machine and hands out at most `--max-tabs` tabs at a time (default 4), each in its own browser context and taken back after `--tab-seconds` (default 120). A lease is an open HTTP request, so a client that exits gives its tab back. With `--proxy-server`, every request goes through the proxy, loopback and WebRTC included. The broker logs Chrome's memory by process type and every lease as JSON lines. `leaseTab` and `serveTabs` are exported from `@rezkam/browser-tools/tab-broker.mjs`, and `connectLeasedTab` connects to a leased tab.
|
|
9
|
+
|
|
3
10
|
## 1.0.4
|
|
4
11
|
|
|
5
12
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -34,12 +34,24 @@ Commands can also run without a global install:
|
|
|
34
34
|
npx @rezkam/browser-tools <command>
|
|
35
35
|
```
|
|
36
36
|
|
|
37
|
-
Browser Tools requires macOS, Google Chrome, and Node.js 20 or newer. GIF recording and review additionally require `ffmpeg` and `ffprobe`:
|
|
37
|
+
Browser Tools requires macOS or Linux, installed Google Chrome or Chromium, and Node.js 20 or newer. GIF recording and review additionally require `ffmpeg` and `ffprobe`:
|
|
38
38
|
|
|
39
39
|
```bash
|
|
40
40
|
brew install ffmpeg
|
|
41
41
|
```
|
|
42
42
|
|
|
43
|
+
## Linux
|
|
44
|
+
|
|
45
|
+
Linux discovers executable Chrome/Chromium launchers on PATH and common distro locations, including the normal snap launcher. Explicit `BROWSER_TOOLS_CHROME_BIN` or `browser.chromeBin` config wins; no browser or OS packages are installed automatically. Use `browser-tools start --headless` without a display; headed mode needs a working X11 or Wayland session. Chromium sandboxing stays enabled. Root, containers, or restrictive AppArmor/user-namespace policies may need operator setup, never an automatic `--no-sandbox` fallback.
|
|
46
|
+
|
|
47
|
+
Profile discovery follows the selected browser family/channel under `${XDG_CONFIG_HOME:-~/.config}` (`google-chrome`, `google-chrome-beta`, `google-chrome-unstable`, or `chromium`); snap Chromium uses `~/snap/chromium/common/chromium`. Override with `BROWSER_TOOLS_CHROME_SOURCE_DIR` or `directories.chromeSourceDir`. `browser-tools config profiles` enumerates Local State into private local config.
|
|
48
|
+
|
|
49
|
+
Profile copying requires `rsync` and `sqlite3` (for default Google identity exclusion). Close the source browser before copying for a consistent snapshot; copying a live SQLite database is not an atomic backup. Copies stay private and source locks are never removed. Run as the same OS user, browser family, and unlocked desktop secret-store session: copied cookies may not decrypt without the original Libsecret/KWallet keyring or D-Bus session. Browser Tools never switches to basic password storage or promises portable logins. Extensions depend on Chromium build/policy and are not guaranteed merely because their files were copied.
|
|
50
|
+
|
|
51
|
+
Snap confinement may deny the default cache or artifact paths. Set `BROWSER_TOOLS_CACHE_DIR` and `BROWSER_TOOLS_ARTIFACT_DIR` to private locations permitted by the installed snap; do not bypass confinement by automatically invoking internal snap executables. Snap launcher discovery is supported, but distro-specific confinement/refresh/keyring behavior requires host validation. Flatpak, Windows, encrypted login migration across users/hosts, and extension execution are not covered by the native compatibility guarantee.
|
|
52
|
+
|
|
53
|
+
Local verification covers sandboxed Linux ARM64 Chromium navigation, eval, screenshots, ownership rejection, fixed-port endpoint identity and cleanup. CI runs unit/contracts and fresh-browser smoke on Linux and macOS (direct executable on hosted macOS); native NSWorkspace routing has a fixture contract, not a real desktop Dock/keychain validation. Real keyrings, headed Linux, snap confinement, and extensions require separate environment-specific verification. Puppeteer API and CLI contracts are retained; no Patchright migration is included.
|
|
54
|
+
|
|
43
55
|
## Quick start
|
|
44
56
|
|
|
45
57
|
```bash
|
|
@@ -67,6 +79,7 @@ Use `--profile "<profile or alias>"` when an existing login is needed, `--sync`
|
|
|
67
79
|
| Visual recording | `record-gif`, `review-gif` | Record an interaction with pre-roll and post-roll frames, then create a contact sheet and metadata report |
|
|
68
80
|
| HTTP capture | `record-har`, `extract-har` | Capture filtered active-tab traffic and derive a compact, non-executable request recipe |
|
|
69
81
|
| Chrome DevTools Protocol | `record-cdp`, `cdp` | Record selected raw protocol events or send one guarded direct method call |
|
|
82
|
+
| Shared tabs | `serve` | Run one lean Chrome for every client on the machine and lease its tabs, a bounded number at a time, each for a bounded time |
|
|
70
83
|
|
|
71
84
|
Run `browser-tools --help` for the command list. Commands with detailed option references, such as `record-har`, `record-cdp`, and `review-gif`, also support command-level `--help`.
|
|
72
85
|
|
|
@@ -93,6 +106,31 @@ import { runBrowserResource } from '@rezkam/browser-tools/resource-helper.mjs';
|
|
|
93
106
|
|
|
94
107
|
The root package export and `@rezkam/browser-tools/browser-control.mjs` both expose the browser lifecycle and connection module. The resource helper export provides the shared lifecycle for generic extractors.
|
|
95
108
|
|
|
109
|
+
### Shared tabs
|
|
110
|
+
|
|
111
|
+
`browser-tools serve` starts a tab broker and prints `{"url": ...}` once it listens. Chrome starts with the first lease, without extensions, component apps, a startup window or background networking.
|
|
112
|
+
|
|
113
|
+
```js
|
|
114
|
+
import { connectLeasedTab } from '@rezkam/browser-tools';
|
|
115
|
+
import { leaseTab } from '@rezkam/browser-tools/tab-broker.mjs';
|
|
116
|
+
|
|
117
|
+
const lease = await leaseTab(brokerUrl, { client: 'quotes' });
|
|
118
|
+
const { browser, page } = await connectLeasedTab(lease);
|
|
119
|
+
try {
|
|
120
|
+
await page.goto('https://example.com/');
|
|
121
|
+
} finally {
|
|
122
|
+
await browser.disconnect();
|
|
123
|
+
lease.release();
|
|
124
|
+
}
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
- At most `--max-tabs` tabs (default 4) are leased at once; later leases wait in order.
|
|
128
|
+
- Each tab has a browser context of its own, so tabs share no cookies or storage, and downloads are refused.
|
|
129
|
+
- A tab is taken back after `--tab-seconds` (default 120), and `lease.ended` then settles with `'expired'`. A client that exits gives its tab back, because a lease is an open HTTP request.
|
|
130
|
+
- With `--proxy-server`, every request goes through the proxy, loopback addresses and WebRTC included.
|
|
131
|
+
- `--until-stdin-closes` stops the broker, and its Chrome, when the parent holding its stdin goes away.
|
|
132
|
+
- On Linux the broker logs Chrome's memory every 15 seconds as a `browser_memory` JSON line on stderr (proportional set size by process type), and each finished lease as a `tab_lease` line with its wait, hold time and how it ended.
|
|
133
|
+
|
|
96
134
|
## Development
|
|
97
135
|
|
|
98
136
|
```bash
|
package/bin/browser-tools.mjs
CHANGED
|
@@ -23,6 +23,7 @@
|
|
|
23
23
|
* browser-tools scrape-page [--port <n>]
|
|
24
24
|
* browser-tools extract-article [--chars <n>] [--port <n>]
|
|
25
25
|
* browser-tools config profiles|active-profiles|task-profile ...
|
|
26
|
+
* browser-tools serve [--max-tabs <n>] [--tab-seconds <n>] [--headless] [--proxy-server <url>] [--until-stdin-closes]
|
|
26
27
|
*/
|
|
27
28
|
|
|
28
29
|
import { spawnSync } from 'node:child_process';
|
|
@@ -49,6 +50,7 @@ const COMMANDS = {
|
|
|
49
50
|
'scrape-page': 'scrape-page.mjs',
|
|
50
51
|
'extract-article': 'extract-article.mjs',
|
|
51
52
|
config: 'config.mjs',
|
|
53
|
+
serve: 'serve.mjs',
|
|
52
54
|
};
|
|
53
55
|
|
|
54
56
|
function printUsage() {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@rezkam/browser-tools",
|
|
3
|
-
"version": "1.0
|
|
3
|
+
"version": "1.1.0",
|
|
4
4
|
"description": "Sandboxed, owner-token-protected Chrome automation for AI agents: control pages, record and review GIFs, capture HAR and raw CDP events, send CDP calls, and manage Chrome profiles.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -22,6 +22,7 @@
|
|
|
22
22
|
".": "./scripts/browser-control.mjs",
|
|
23
23
|
"./browser-control.mjs": "./scripts/browser-control.mjs",
|
|
24
24
|
"./resource-helper.mjs": "./scripts/resource-helper.mjs",
|
|
25
|
+
"./tab-broker.mjs": "./scripts/tab-broker.mjs",
|
|
25
26
|
"./package.json": "./package.json"
|
|
26
27
|
},
|
|
27
28
|
"files": [
|
|
@@ -41,7 +42,8 @@
|
|
|
41
42
|
"validate": "npm test && npm run test:e2e"
|
|
42
43
|
},
|
|
43
44
|
"dependencies": {
|
|
44
|
-
"puppeteer-core": "^24.42.0"
|
|
45
|
+
"puppeteer-core": "^24.42.0",
|
|
46
|
+
"ws": "^8.22.0"
|
|
45
47
|
},
|
|
46
48
|
"devDependencies": {
|
|
47
49
|
"@changesets/cli": "^2.31.0"
|
|
@@ -17,14 +17,16 @@ import { homedir, tmpdir } from 'node:os';
|
|
|
17
17
|
import { delimiter, dirname, join, resolve } from 'node:path';
|
|
18
18
|
import { fileURLToPath } from 'node:url';
|
|
19
19
|
import puppeteer from 'puppeteer-core';
|
|
20
|
+
import WebSocket from 'ws';
|
|
21
|
+
import { discoverChromeBinary, discoverChromeSourceDir, chromePlatformToken } from './browser-platform.mjs';
|
|
20
22
|
|
|
21
23
|
export const DEFAULT_PORT = 9222;
|
|
22
24
|
export const CONNECT_PROBE_TIMEOUT_MS = 3000;
|
|
23
25
|
export const PORT_OCCUPANCY_PROBE_TIMEOUT_MS = 250;
|
|
24
26
|
export const CHROME_READY_TIMEOUT_MS = 15000;
|
|
25
27
|
export const CHROME_READY_PROBE_TIMEOUT_MS = 250;
|
|
26
|
-
export const DEFAULT_CHROME_BIN = '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome';
|
|
27
|
-
export const DEFAULT_CHROME_SRC = join(homedir(), '
|
|
28
|
+
export const DEFAULT_CHROME_BIN = discoverChromeBinary() || (process.platform === 'linux' ? 'google-chrome' : '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome');
|
|
29
|
+
export const DEFAULT_CHROME_SRC = discoverChromeSourceDir({ chromeBin: DEFAULT_CHROME_BIN }) || join(homedir(), '.config', 'google-chrome');
|
|
28
30
|
export const DEFAULT_CACHE_DIR = join(homedir(), '.cache', 'pi-browser-tools');
|
|
29
31
|
export const DEFAULT_ARTIFACT_DIR = tmpdir();
|
|
30
32
|
|
|
@@ -84,11 +86,12 @@ export function expandHomePath(value) {
|
|
|
84
86
|
export function browserToolsRuntimeConfig({ configDir = browserToolsConfigDir() } = {}) {
|
|
85
87
|
const effectiveConfigDir = resolveBrowserToolsConfigDir(configDir);
|
|
86
88
|
const config = safeReadJson(browserToolsConfigFile(effectiveConfigDir)) || {};
|
|
89
|
+
const chromeBin = expandHomePath(process.env.BROWSER_TOOLS_CHROME_BIN || config.browser?.chromeBin || config.chromeBin || discoverChromeBinary() || DEFAULT_CHROME_BIN);
|
|
87
90
|
return {
|
|
88
91
|
configDir: effectiveConfigDir,
|
|
89
92
|
configFile: browserToolsConfigFile(effectiveConfigDir),
|
|
90
|
-
chromeBin
|
|
91
|
-
chromeSourceDir: expandHomePath(process.env.BROWSER_TOOLS_CHROME_SOURCE_DIR || config.directories?.chromeSourceDir || config.sourceDir || DEFAULT_CHROME_SRC),
|
|
93
|
+
chromeBin,
|
|
94
|
+
chromeSourceDir: expandHomePath(process.env.BROWSER_TOOLS_CHROME_SOURCE_DIR || config.directories?.chromeSourceDir || config.sourceDir || discoverChromeSourceDir({ chromeBin }) || DEFAULT_CHROME_SRC),
|
|
92
95
|
cacheDir: expandHomePath(process.env.BROWSER_TOOLS_CACHE_DIR || config.directories?.cacheDir || DEFAULT_CACHE_DIR),
|
|
93
96
|
artifactDir: expandHomePath(process.env.BROWSER_TOOLS_ARTIFACT_DIR || config.directories?.artifactDir || DEFAULT_ARTIFACT_DIR),
|
|
94
97
|
maxBrowsers: config.browser?.maxBrowsers ?? config.maxBrowsers ?? null,
|
|
@@ -379,7 +382,7 @@ export async function browserWSEndpoint(port = DEFAULT_PORT, timeoutMs = CONNECT
|
|
|
379
382
|
const controller = new AbortController();
|
|
380
383
|
const timer = setTimeout(() => controller.abort(), timeoutMs);
|
|
381
384
|
try {
|
|
382
|
-
const res = await fetch(`http://
|
|
385
|
+
const res = await fetch(`http://127.0.0.1:${normalizedPort}/json/version`, { signal: controller.signal });
|
|
383
386
|
if (!res.ok) return null;
|
|
384
387
|
const data = await res.json();
|
|
385
388
|
return data.webSocketDebuggerUrl ?? null;
|
|
@@ -437,6 +440,47 @@ export function devToolsActivePortMatches(userDataDir, endpoint) {
|
|
|
437
440
|
}
|
|
438
441
|
}
|
|
439
442
|
|
|
443
|
+
// Fixed-port Chromium need not write DevToolsActivePort. A loopback URL alone is
|
|
444
|
+
// not ownership proof: ask the endpoint for its actual browser launch arguments.
|
|
445
|
+
export async function launchedEndpointMatches({ port, endpoint, state, timeoutMs = 3000 }) {
|
|
446
|
+
let url;
|
|
447
|
+
try { url = new URL(endpoint); } catch { return false; }
|
|
448
|
+
if (url.hash || url.username || url.password || url.protocol !== 'ws:' || !['127.0.0.1', 'localhost', '[::1]'].includes(url.hostname)
|
|
449
|
+
|| Number(url.port) !== Number(port) || !url.pathname.startsWith('/devtools/browser/')) return false;
|
|
450
|
+
if (devToolsActivePortMatches(state?.userDataDir, endpoint)) return true;
|
|
451
|
+
if (!state?.managedToken || !state?.userDataDir) return false;
|
|
452
|
+
return new Promise((resolve) => {
|
|
453
|
+
let socket;
|
|
454
|
+
try { socket = new WebSocket(endpoint, { followRedirects: false, handshakeTimeout: timeoutMs, maxPayload: 1024 * 1024 }); }
|
|
455
|
+
catch { resolve(false); return; }
|
|
456
|
+
let settled = false;
|
|
457
|
+
const finish = (ok) => {
|
|
458
|
+
if (settled) return;
|
|
459
|
+
settled = true;
|
|
460
|
+
clearTimeout(timer);
|
|
461
|
+
socket.terminate();
|
|
462
|
+
resolve(ok);
|
|
463
|
+
};
|
|
464
|
+
const timer = setTimeout(() => finish(false), timeoutMs);
|
|
465
|
+
socket.on('error', () => finish(false));
|
|
466
|
+
socket.on('close', () => finish(false));
|
|
467
|
+
socket.on('open', () => socket.send(JSON.stringify({ id: 1, method: 'Browser.getBrowserCommandLine' })));
|
|
468
|
+
socket.on('message', (data) => {
|
|
469
|
+
try {
|
|
470
|
+
const message = JSON.parse(data.toString());
|
|
471
|
+
if (message.id !== 1) return;
|
|
472
|
+
const args = message.result?.arguments;
|
|
473
|
+
finish(Array.isArray(args) && [
|
|
474
|
+
'--enable-automation',
|
|
475
|
+
'--remote-debugging-port=' + port,
|
|
476
|
+
'--user-data-dir=' + state.userDataDir,
|
|
477
|
+
'--pi-browser-tools-managed=' + state.managedToken,
|
|
478
|
+
].every(arg => args.includes(arg)));
|
|
479
|
+
} catch { finish(false); }
|
|
480
|
+
});
|
|
481
|
+
});
|
|
482
|
+
}
|
|
483
|
+
|
|
440
484
|
export async function waitForChromeEndpoint(
|
|
441
485
|
port = DEFAULT_PORT,
|
|
442
486
|
{ timeoutMs = CHROME_READY_TIMEOUT_MS, intervalMs = 250, probeTimeoutMs = CHROME_READY_PROBE_TIMEOUT_MS } = {},
|
|
@@ -821,6 +865,7 @@ export function syncChromeProfile(profileName, { force = false, port = DEFAULT_P
|
|
|
821
865
|
const cachedMatches = cachedState
|
|
822
866
|
&& typeof cachedState.includeGoogle === 'boolean'
|
|
823
867
|
&& cachedState.includeGoogle === includeGoogle
|
|
868
|
+
&& cachedState.sourceDir && resolve(cachedState.sourceDir) === resolve(sourceDir)
|
|
824
869
|
&& (includeGoogle || cachedState.googleStripOk === true);
|
|
825
870
|
if (!force && cachedMatches && profileCopyReady(profileName, destDir)) {
|
|
826
871
|
removeChromeProfileLocks(destDir);
|
|
@@ -830,11 +875,12 @@ export function syncChromeProfile(profileName, { force = false, port = DEFAULT_P
|
|
|
830
875
|
const commands = profileSyncRsyncCommands(profileName, { sourceDir, destDir, checkExists: true });
|
|
831
876
|
|
|
832
877
|
rmSync(destDir, { recursive: true, force: true });
|
|
833
|
-
mkdirSync(destDir, { recursive: true });
|
|
878
|
+
mkdirSync(destDir, { recursive: true, mode: 0o700 });
|
|
879
|
+
chmodSync(destDir, 0o700);
|
|
834
880
|
const results = [];
|
|
835
881
|
const copiedItems = [];
|
|
836
882
|
for (const command of commands) {
|
|
837
|
-
mkdirSync(command.dest, { recursive: true });
|
|
883
|
+
mkdirSync(command.dest, { recursive: true, mode: 0o700 });
|
|
838
884
|
const result = spawnSync('rsync', command.args, { stdio: 'inherit' });
|
|
839
885
|
if (result.error) throw result.error;
|
|
840
886
|
if (result.signal) throw new Error(`rsync failed for ${command.item}: terminated by signal ${result.signal}`);
|
|
@@ -968,6 +1014,8 @@ export function chromeLaunchArgs({ port = DEFAULT_PORT, profileName = null, mana
|
|
|
968
1014
|
const args = [
|
|
969
1015
|
`--remote-debugging-port=${normalizedPort}`,
|
|
970
1016
|
`--user-data-dir=${launchUserDataDir}`,
|
|
1017
|
+
'--remote-debugging-address=127.0.0.1',
|
|
1018
|
+
'--enable-automation',
|
|
971
1019
|
'--no-first-run',
|
|
972
1020
|
'--no-default-browser-check',
|
|
973
1021
|
'--disable-sync',
|
|
@@ -1000,7 +1048,7 @@ export function detectChromeMajorVersion(chromeBin = browserToolsChromeBin()) {
|
|
|
1000
1048
|
// so this matches what a windowed macOS Chrome sends. It carries no machine-specific data.
|
|
1001
1049
|
export function chromeUserAgentForMajor(major) {
|
|
1002
1050
|
if (!Number.isInteger(major) || major < 1) return null;
|
|
1003
|
-
return `Mozilla/5.0 (
|
|
1051
|
+
return `Mozilla/5.0 (${chromePlatformToken()}) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/${major}.0.0.0 Safari/537.36`;
|
|
1004
1052
|
}
|
|
1005
1053
|
|
|
1006
1054
|
export function headlessUserAgent(chromeBin = browserToolsChromeBin()) {
|
|
@@ -1134,7 +1182,7 @@ export function managedBrowserIdentityMatches(expected, current) {
|
|
|
1134
1182
|
export function verifyManagedChromePid(pid, { cacheDir = browserToolsCacheDir(), chromeBin = browserToolsChromeBin() } = {}) {
|
|
1135
1183
|
const command = commandForPid(pid);
|
|
1136
1184
|
if (!command) return null;
|
|
1137
|
-
return managedChromeProcessFromCommand({ pid, command, cacheDir, chromeBin });
|
|
1185
|
+
return managedChromeProcessFromCommand({ pid, command: command.replaceAll('\0', ' ').trim(), cacheDir, chromeBin });
|
|
1138
1186
|
}
|
|
1139
1187
|
|
|
1140
1188
|
// The live inventory of managed browsers. This reads the process table rather than the lifecycle
|
|
@@ -1477,6 +1525,7 @@ export function managedBrowserReuseDecision({
|
|
|
1477
1525
|
// Adopting a browser built from a different profile (or a fresh, signed-out clone) would run the
|
|
1478
1526
|
// caller's automation against the wrong account.
|
|
1479
1527
|
if ((state?.profileName || null) !== (profileName || null)) return fail('profile-mismatch');
|
|
1528
|
+
if (profileName && (!state.profileSourceDir || resolve(state.profileSourceDir) !== resolve(browserToolsChromeSourceDir()))) return fail('profile-source-mismatch');
|
|
1480
1529
|
// Headless is fixed at launch, so a windowed request cannot be served by a headless browser.
|
|
1481
1530
|
if (Boolean(state?.headless) !== Boolean(headless)) return fail('headless-mismatch');
|
|
1482
1531
|
return { ok: true, ownerId: ownership.ownerId, state };
|
|
@@ -1579,6 +1628,11 @@ export function launchChrome({ port = DEFAULT_PORT, profileName = null, userData
|
|
|
1579
1628
|
// A stale marker must not make a later launch appear to own a competing endpoint.
|
|
1580
1629
|
rmSync(join(launchUserDataDir, 'DevToolsActivePort'), { force: true });
|
|
1581
1630
|
assertChromeBinaryLaunchable(chromeBin);
|
|
1631
|
+
if (process.platform === 'linux' && !headless && !process.env.DISPLAY && !process.env.WAYLAND_DISPLAY) {
|
|
1632
|
+
throw new Error('No Linux display found. Pass --headless (API: headless: true), or provide a working DISPLAY/WAYLAND_DISPLAY.');
|
|
1633
|
+
}
|
|
1634
|
+
mkdirSync(launchUserDataDir, { recursive: true, mode: 0o700 });
|
|
1635
|
+
chmodSync(launchUserDataDir, 0o700);
|
|
1582
1636
|
const pidFile = pidFileForPort(normalizedPort);
|
|
1583
1637
|
const stateFile = stateFileForPort(normalizedPort);
|
|
1584
1638
|
let proc = headless ? launchHeadlessChromeApp({ chromeBin, args }) : null;
|
|
@@ -1603,6 +1657,7 @@ export function launchChrome({ port = DEFAULT_PORT, profileName = null, userData
|
|
|
1603
1657
|
pid: proc.pid,
|
|
1604
1658
|
port: normalizedPort,
|
|
1605
1659
|
profileName: profileName || null,
|
|
1660
|
+
profileSourceDir: profileName ? browserToolsChromeSourceDir() : null,
|
|
1606
1661
|
userDataDir: launchUserDataDir,
|
|
1607
1662
|
headless: Boolean(headless),
|
|
1608
1663
|
includeGoogle: Boolean(includeGoogle),
|
|
@@ -1761,6 +1816,9 @@ export async function startChrome({
|
|
|
1761
1816
|
}
|
|
1762
1817
|
// A configuration mismatch is not an ownership problem: the caller owns this browser, it
|
|
1763
1818
|
// was just built differently. Say so, rather than sending them to look for a token.
|
|
1819
|
+
if (reuse.reason === 'profile-source-mismatch') {
|
|
1820
|
+
throw new Error('Managed Chrome profile source differs or was not recorded by this version. Stop it with the owner token and --clean, then restart for the requested source profile.');
|
|
1821
|
+
}
|
|
1764
1822
|
if (reuse.reason === 'profile-mismatch') {
|
|
1765
1823
|
throw new Error(
|
|
1766
1824
|
`Chrome on :${normalizedPort} was started with profile ${state?.profileName ? `"${state.profileName}"` : '(fresh, no profile)'}, but this start requested ${resolvedProfileName ? `"${resolvedProfileName}"` : '(fresh, no profile)'}. Reusing it would run against the wrong account. Stop it with scripts/stop.mjs --clean and start again, or pick a different --port.`,
|
|
@@ -1916,11 +1974,11 @@ export async function startChrome({
|
|
|
1916
1974
|
const launchedEndpoint = await waitForChromeEndpoint(normalizedPort, { timeoutMs: CHROME_READY_TIMEOUT_MS });
|
|
1917
1975
|
if (!launchedEndpoint) {
|
|
1918
1976
|
stopChrome({ port: normalizedPort, ownerToken: effectiveOwnerToken, ignorePortLock: true });
|
|
1919
|
-
throw new Error(`Failed to connect to Chrome after ${Math.round(CHROME_READY_TIMEOUT_MS / 1000)}s
|
|
1977
|
+
throw new Error(`Failed to connect to Chrome after ${Math.round(CHROME_READY_TIMEOUT_MS / 1000)}s. Check the executable, Linux sandbox/user-namespace policy, display (or --headless), and profile permissions. Snap confinement may require BROWSER_TOOLS_CACHE_DIR and BROWSER_TOOLS_ARTIFACT_DIR inside its accessible directories. Browser Tools never disables the sandbox automatically.`);
|
|
1920
1978
|
}
|
|
1921
1979
|
const launchedState = readManagedStateForPort(normalizedPort);
|
|
1922
1980
|
const launchedSafety = managedBrowserSafetyForPort(normalizedPort);
|
|
1923
|
-
if (!launchedSafety.ok || Number(launchedState?.pid) !== Number(proc?.pid) || !
|
|
1981
|
+
if (!launchedSafety.ok || Number(launchedState?.pid) !== Number(proc?.pid) || !await launchedEndpointMatches({ port: normalizedPort, endpoint: launchedEndpoint, state: launchedState })) {
|
|
1924
1982
|
stopChrome({ port: normalizedPort, ownerToken: effectiveOwnerToken, ignorePortLock: true });
|
|
1925
1983
|
throw new Error(
|
|
1926
1984
|
`Chrome reported ready on :${normalizedPort}, but the responding process was not verified as the managed launch (${launchedSafety.reason || 'pid-mismatch'}).`,
|
|
@@ -1987,7 +2045,7 @@ export function stopChrome({ port = DEFAULT_PORT, clean = false, dryRun = false,
|
|
|
1987
2045
|
rmSync(paths.stateFile, { force: true });
|
|
1988
2046
|
let cleaned = false;
|
|
1989
2047
|
if (clean && isBrowserToolsUserDataDir(state?.userDataDir)) {
|
|
1990
|
-
rmSync(state.userDataDir, { recursive: true, force: true });
|
|
2048
|
+
rmSync(state.userDataDir, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 });
|
|
1991
2049
|
rmSync(profileSyncStateFileForPort(normalizedPort), { force: true });
|
|
1992
2050
|
cleaned = true;
|
|
1993
2051
|
}
|
|
@@ -2079,7 +2137,7 @@ export function stopChrome({ port = DEFAULT_PORT, clean = false, dryRun = false,
|
|
|
2079
2137
|
rmSync(paths.stateFile, { force: true });
|
|
2080
2138
|
let cleaned = false;
|
|
2081
2139
|
if (clean && isBrowserToolsUserDataDir(state?.userDataDir)) {
|
|
2082
|
-
rmSync(state.userDataDir, { recursive: true, force: true });
|
|
2140
|
+
rmSync(state.userDataDir, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 });
|
|
2083
2141
|
rmSync(profileSyncStateFileForPort(normalizedPort), { force: true });
|
|
2084
2142
|
cleaned = true;
|
|
2085
2143
|
}
|
|
@@ -2165,6 +2223,19 @@ function readManagedState(stateFile) {
|
|
|
2165
2223
|
}
|
|
2166
2224
|
|
|
2167
2225
|
function commandForPid(pid) {
|
|
2226
|
+
if (process.platform === 'linux') {
|
|
2227
|
+
try {
|
|
2228
|
+
const command = readFileSync(`/proc/${pid}/cmdline`, 'utf8');
|
|
2229
|
+
// Chromium rewrites its process title into argv[0], leaving only trailing NULs.
|
|
2230
|
+
// Retain exact boundaries for ordinary argv; parse the rewritten title as ps does.
|
|
2231
|
+
if (command) return command.split('\0').filter(Boolean).length === 1 ? command.replaceAll('\0', '').trim() : command;
|
|
2232
|
+
if (!processExists(pid)) return null;
|
|
2233
|
+
throw new Error(`Cannot inspect live browser process ${pid}`);
|
|
2234
|
+
} catch (error) {
|
|
2235
|
+
if (error.code === 'ENOENT' || error.code === 'ESRCH') return null;
|
|
2236
|
+
throw error;
|
|
2237
|
+
}
|
|
2238
|
+
}
|
|
2168
2239
|
const result = spawnSync('ps', ['-p', String(pid), '-o', 'command='], { encoding: 'utf-8' });
|
|
2169
2240
|
if (result.error || result.status !== 0) return null;
|
|
2170
2241
|
const command = result.stdout.trim();
|
|
@@ -2194,6 +2265,12 @@ function processStartIdentity(pid) {
|
|
|
2194
2265
|
}
|
|
2195
2266
|
|
|
2196
2267
|
function processIsZombie(pid) {
|
|
2268
|
+
if (process.platform === 'linux') {
|
|
2269
|
+
try {
|
|
2270
|
+
const stat = readFileSync(`/proc/${pid}/stat`, 'utf8');
|
|
2271
|
+
return stat.slice(stat.lastIndexOf(')') + 2).startsWith('Z');
|
|
2272
|
+
} catch { return false; }
|
|
2273
|
+
}
|
|
2197
2274
|
const result = spawnSync('ps', ['-p', String(pid), '-o', 'state='], { encoding: 'utf-8' });
|
|
2198
2275
|
if (result.error || result.status !== 0) return false;
|
|
2199
2276
|
return result.stdout.trim().startsWith('Z');
|
|
@@ -2272,9 +2349,18 @@ export function managedChromeCommandSafety({ pid, port, state, command }) {
|
|
|
2272
2349
|
const expectedToken = `--pi-browser-tools-managed=${state.managedToken}`;
|
|
2273
2350
|
const expectedUserDataDir = `--user-data-dir=${state.userDataDir}`;
|
|
2274
2351
|
|
|
2275
|
-
|
|
2276
|
-
|
|
2277
|
-
|
|
2352
|
+
const argv = command.includes('\0') ? command.split('\0') : null;
|
|
2353
|
+
const hasArgument = (arg) => {
|
|
2354
|
+
if (argv) return argv.includes(arg);
|
|
2355
|
+
const start = command.indexOf(arg);
|
|
2356
|
+
if (start < 0) return false;
|
|
2357
|
+
const end = start + arg.length;
|
|
2358
|
+
return (start === 0 || /\s/.test(command[start - 1]))
|
|
2359
|
+
&& (end === command.length || /\s/.test(command[end]));
|
|
2360
|
+
};
|
|
2361
|
+
if (!hasArgument(expectedDebugPort)) return { ok: false, reason: 'debug-port-mismatch', command };
|
|
2362
|
+
if (!hasArgument(expectedUserDataDir)) return { ok: false, reason: 'user-data-dir-mismatch', command };
|
|
2363
|
+
if (!hasArgument(expectedToken)) return { ok: false, reason: 'managed-token-mismatch', command };
|
|
2278
2364
|
|
|
2279
2365
|
return { ok: true, command };
|
|
2280
2366
|
}
|
|
@@ -2292,12 +2378,35 @@ export async function connectBrowser(port = DEFAULT_PORT, options = {}) {
|
|
|
2292
2378
|
throw new Error(`Refusing to connect to Browser Tools Chrome on :${normalizedPort}: ${ownership.reason}`);
|
|
2293
2379
|
}
|
|
2294
2380
|
return puppeteer.connect({
|
|
2295
|
-
browserURL: `http://
|
|
2381
|
+
browserURL: `http://127.0.0.1:${normalizedPort}`,
|
|
2296
2382
|
defaultViewport: null,
|
|
2297
2383
|
...puppeteerOptions,
|
|
2298
2384
|
});
|
|
2299
2385
|
}
|
|
2300
2386
|
|
|
2387
|
+
/** Connects to a browser this process started itself, which needs no managed-browser ownership check. */
|
|
2388
|
+
export async function connectOwnBrowser(browserWSEndpoint, options = {}) {
|
|
2389
|
+
return puppeteer.connect({ browserWSEndpoint, defaultViewport: null, ...options });
|
|
2390
|
+
}
|
|
2391
|
+
|
|
2392
|
+
/** Connects to the one tab a tab broker leased (see tab-broker.mjs), seeing no other tab of the browser. */
|
|
2393
|
+
export async function connectLeasedTab({ cdpUrl, tabUrl }, options = {}) {
|
|
2394
|
+
if (typeof cdpUrl !== 'string' || !cdpUrl.startsWith('ws://')) throw new Error('cdpUrl must be a ws:// address');
|
|
2395
|
+
if (typeof tabUrl !== 'string' || !tabUrl.startsWith('about:blank#')) throw new Error('tabUrl must be a leased tab address');
|
|
2396
|
+
const browser = await puppeteer.connect({
|
|
2397
|
+
browserWSEndpoint: cdpUrl,
|
|
2398
|
+
defaultViewport: null,
|
|
2399
|
+
...options,
|
|
2400
|
+
targetFilter: (target) => target.url() === tabUrl,
|
|
2401
|
+
});
|
|
2402
|
+
const [page] = await browser.pages();
|
|
2403
|
+
if (!page) {
|
|
2404
|
+
await browser.disconnect();
|
|
2405
|
+
throw new Error('The leased tab is gone');
|
|
2406
|
+
}
|
|
2407
|
+
return { browser, page };
|
|
2408
|
+
}
|
|
2409
|
+
|
|
2301
2410
|
export async function withBrowser(port, callback, options = {}) {
|
|
2302
2411
|
const browser = await connectBrowser(port, options);
|
|
2303
2412
|
try {
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
import { accessSync, constants, readFileSync, realpathSync, statSync } from 'node:fs';
|
|
2
|
+
import { homedir } from 'node:os';
|
|
3
|
+
import { basename, isAbsolute, join } from 'node:path';
|
|
4
|
+
|
|
5
|
+
const MAC_CHROME = '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome';
|
|
6
|
+
const LINUX_NAMES = [
|
|
7
|
+
'google-chrome', 'google-chrome-stable', 'google-chrome-beta',
|
|
8
|
+
'google-chrome-unstable', 'chromium', 'chromium-browser',
|
|
9
|
+
];
|
|
10
|
+
const LINUX_LOCATIONS = ['/usr/bin', '/usr/local/bin', '/opt/bin']
|
|
11
|
+
.flatMap((directory) => LINUX_NAMES.map((name) => join(directory, name))).concat([
|
|
12
|
+
// Use the supported snap launcher, never its internal binary.
|
|
13
|
+
'/snap/bin/chromium',
|
|
14
|
+
'/opt/google/chrome/chrome', '/opt/google/chrome-beta/chrome',
|
|
15
|
+
'/opt/google/chrome-unstable/chrome',
|
|
16
|
+
'/usr/lib/chromium/chromium', '/usr/lib64/chromium-browser/chromium-browser',
|
|
17
|
+
'/usr/lib/chromium-browser/chromium-browser', '/usr/lib64/chromium/chromium',
|
|
18
|
+
]);
|
|
19
|
+
|
|
20
|
+
function executable(file) {
|
|
21
|
+
try {
|
|
22
|
+
accessSync(file, constants.X_OK);
|
|
23
|
+
return statSync(file).isFile();
|
|
24
|
+
} catch { return false; }
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
/** Read-only discovery: executable Linux launcher or null if absent.
|
|
28
|
+
* macOS preserves its historical default even when not installed.
|
|
29
|
+
* Explicit configuration/env overrides belong to the caller.
|
|
30
|
+
*/
|
|
31
|
+
export function discoverChromeBinary({ platform = process.platform, env = process.env, home = homedir() } = {}) {
|
|
32
|
+
if (platform === 'darwin') return MAC_CHROME;
|
|
33
|
+
if (platform !== 'linux') return null;
|
|
34
|
+
void home; // Do not infer installations from arbitrary downloads/caches.
|
|
35
|
+
const directories = (env.PATH || '').split(':').filter(Boolean);
|
|
36
|
+
for (const name of LINUX_NAMES) {
|
|
37
|
+
for (const directory of directories) {
|
|
38
|
+
const candidate = join(directory, name);
|
|
39
|
+
if (executable(candidate)) return candidate;
|
|
40
|
+
}
|
|
41
|
+
}
|
|
42
|
+
return LINUX_LOCATIONS.find(executable) || null;
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
function directoryExists(directory) {
|
|
46
|
+
try { return statSync(directory).isDirectory(); } catch { return false; }
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
function snapChromium(binary, env = process.env) {
|
|
50
|
+
if (!binary.includes('/')) {
|
|
51
|
+
binary = (env.PATH || '').split(':').filter(Boolean).map(dir => join(dir, binary)).find(executable) || binary;
|
|
52
|
+
}
|
|
53
|
+
if (binary === '/snap/bin/chromium' || binary.includes('/snap/chromium/')) return true;
|
|
54
|
+
try {
|
|
55
|
+
const target = realpathSync(binary);
|
|
56
|
+
if (target === '/usr/bin/snap' || target.includes('/snap/chromium/')) return true;
|
|
57
|
+
// Ubuntu's chromium-browser compatibility launcher delegates to snap.
|
|
58
|
+
if (statSync(binary).size < 64 * 1024) {
|
|
59
|
+
return /(?:\/snap\/bin\/chromium|\bsnap\s+run\s+chromium\b)/.test(readFileSync(binary, 'utf8'));
|
|
60
|
+
}
|
|
61
|
+
} catch { /* Missing or unreadable launchers retain predictable defaults. */ }
|
|
62
|
+
return false;
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
/** User-data root for the selected browser family/channel; never creates it.
|
|
66
|
+
* Never substitutes another browser family's existing profile.
|
|
67
|
+
*/
|
|
68
|
+
export function discoverChromeSourceDir({ chromeBin, platform = process.platform, env = process.env, home = homedir() } = {}) {
|
|
69
|
+
if (platform === 'darwin') return join(home, 'Library', 'Application Support', 'Google', 'Chrome');
|
|
70
|
+
if (platform !== 'linux') return null;
|
|
71
|
+
const binary = chromeBin || discoverChromeBinary({ platform, env, home }) || 'google-chrome';
|
|
72
|
+
const name = basename(binary).toLowerCase();
|
|
73
|
+
const configRoot = env.XDG_CONFIG_HOME && isAbsolute(env.XDG_CONFIG_HOME)
|
|
74
|
+
? env.XDG_CONFIG_HOME : join(home, '.config');
|
|
75
|
+
if (name.includes('chromium') || binary.includes('/snap/chromium/')) {
|
|
76
|
+
const native = join(configRoot, 'chromium');
|
|
77
|
+
const snap = join(home, 'snap', 'chromium', 'common', 'chromium');
|
|
78
|
+
if (snapChromium(binary, env)) return snap;
|
|
79
|
+
// Distro launchers can delegate to snap; stay within the Chromium family.
|
|
80
|
+
return !directoryExists(native) && directoryExists(snap) ? snap : native;
|
|
81
|
+
}
|
|
82
|
+
const channel = /(?:google-chrome|chrome)[-/]beta(?:[/]|$)/.test(binary) ? '-beta'
|
|
83
|
+
: /(?:google-chrome|chrome)[-/](?:unstable|dev)(?:[/]|$)/.test(binary) ? '-unstable' : '';
|
|
84
|
+
return join(configRoot, 'google-chrome' + channel);
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
/** User-agent platform component, not a full User-Agent string. */
|
|
88
|
+
export function chromePlatformToken({ platform = process.platform, arch = process.arch } = {}) {
|
|
89
|
+
if (platform === 'darwin') return 'Macintosh; Intel Mac OS X 10_15_7';
|
|
90
|
+
if (platform === 'win32') return 'Windows NT 10.0; Win64; x64';
|
|
91
|
+
if (platform === 'linux') {
|
|
92
|
+
const machine = { x64: 'x86_64', arm64: 'aarch64', ia32: 'i686', arm: 'armv7l' }[arch] || arch;
|
|
93
|
+
return 'X11; Linux ' + machine;
|
|
94
|
+
}
|
|
95
|
+
return 'X11; ' + platform + ' ' + arch;
|
|
96
|
+
}
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Serves tabs of one shared Chrome to every client that leases one (see tab-broker.mjs).
|
|
4
|
+
*
|
|
5
|
+
* Usage: serve.mjs [--max-tabs <n>] [--tab-seconds <n>] [--headless] [--proxy-server <url>]
|
|
6
|
+
* [--chrome-arg <arg>]... [--port <n>] [--until-stdin-closes]
|
|
7
|
+
*
|
|
8
|
+
* Prints {"url": ...} on stdout once it listens, and one JSON log line per event on stderr.
|
|
9
|
+
*/
|
|
10
|
+
|
|
11
|
+
import { browserToolsChromeBin, hasFlag, parsePositiveIntegerOption, requiredOptionValue } from './browser-control.mjs';
|
|
12
|
+
import { DEFAULT_MAX_TABS, DEFAULT_TAB_SECONDS, serveTabs } from './tab-broker.mjs';
|
|
13
|
+
|
|
14
|
+
const args = process.argv.slice(2);
|
|
15
|
+
const chromeArgs = args.flatMap((arg, index) => (args[index - 1] === '--chrome-arg' ? [arg] : []));
|
|
16
|
+
|
|
17
|
+
const served = await serveTabs({
|
|
18
|
+
chromeBin: browserToolsChromeBin(),
|
|
19
|
+
chromeArgs,
|
|
20
|
+
headless: hasFlag(args, '--headless'),
|
|
21
|
+
proxyServer: requiredOptionValue(args, '--proxy-server'),
|
|
22
|
+
maxTabs: parsePositiveIntegerOption(args, '--max-tabs', DEFAULT_MAX_TABS),
|
|
23
|
+
tabSeconds: parsePositiveIntegerOption(args, '--tab-seconds', DEFAULT_TAB_SECONDS),
|
|
24
|
+
port: parsePositiveIntegerOption(args, '--port', 0),
|
|
25
|
+
});
|
|
26
|
+
process.stdout.write(`${JSON.stringify({ url: served.url })}\n`);
|
|
27
|
+
|
|
28
|
+
let stopping = false;
|
|
29
|
+
async function stop() {
|
|
30
|
+
if (stopping) return;
|
|
31
|
+
stopping = true;
|
|
32
|
+
await served.close();
|
|
33
|
+
process.exit(0);
|
|
34
|
+
}
|
|
35
|
+
process.on('SIGTERM', stop);
|
|
36
|
+
process.on('SIGINT', stop);
|
|
37
|
+
// A parent that holds our stdin and dies, even by SIGKILL, closes it, so the browser goes with it.
|
|
38
|
+
if (hasFlag(args, '--until-stdin-closes')) {
|
|
39
|
+
process.stdin.on('end', stop);
|
|
40
|
+
process.stdin.on('close', stop);
|
|
41
|
+
process.stdin.resume();
|
|
42
|
+
}
|
|
@@ -0,0 +1,351 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* One Chrome shared by every client on the machine, handing out at most `maxTabs` tabs at a time.
|
|
3
|
+
*
|
|
4
|
+
* A client leases a tab with an HTTP request that the broker answers with one JSON line once a tab is
|
|
5
|
+
* free and then holds open. The tab lives in a browser context of its own, so tabs share no cookies or
|
|
6
|
+
* storage, and the context is disposed when the client closes the connection (or exits), or when the
|
|
7
|
+
* tab has been held for `tabSeconds`.
|
|
8
|
+
*/
|
|
9
|
+
|
|
10
|
+
import { spawn } from 'node:child_process';
|
|
11
|
+
import { randomUUID } from 'node:crypto';
|
|
12
|
+
import { existsSync, mkdtempSync, readdirSync, readFileSync, rmSync } from 'node:fs';
|
|
13
|
+
import { createServer, get as httpGet } from 'node:http';
|
|
14
|
+
import { tmpdir } from 'node:os';
|
|
15
|
+
import { join } from 'node:path';
|
|
16
|
+
import { connectOwnBrowser } from './browser-control.mjs';
|
|
17
|
+
|
|
18
|
+
export const DEFAULT_MAX_TABS = 4;
|
|
19
|
+
export const DEFAULT_TAB_SECONDS = 120;
|
|
20
|
+
export const DEFAULT_SAMPLE_SECONDS = 15;
|
|
21
|
+
const CHROME_START_TIMEOUT_MS = 30_000;
|
|
22
|
+
const CHROME_STOP_TIMEOUT_MS = 5_000;
|
|
23
|
+
|
|
24
|
+
// Everything a fresh profile would start in the background is off: tabs are the only work.
|
|
25
|
+
export const LEAN_CHROME_ARGS = [
|
|
26
|
+
'--no-first-run',
|
|
27
|
+
'--no-default-browser-check',
|
|
28
|
+
'--no-startup-window',
|
|
29
|
+
'--disable-sync',
|
|
30
|
+
'--disable-extensions',
|
|
31
|
+
'--disable-component-extensions-with-background-pages',
|
|
32
|
+
'--disable-component-update',
|
|
33
|
+
'--disable-default-apps',
|
|
34
|
+
'--disable-background-networking',
|
|
35
|
+
'--disable-domain-reliability',
|
|
36
|
+
'--disable-client-side-phishing-detection',
|
|
37
|
+
'--disable-breakpad',
|
|
38
|
+
'--disable-crash-reporter',
|
|
39
|
+
'--no-service-autorun',
|
|
40
|
+
'--password-store=basic',
|
|
41
|
+
'--mute-audio',
|
|
42
|
+
'--disable-blink-features=AutomationControlled',
|
|
43
|
+
// A closed tab's renderer would otherwise stay alive for the back-forward cache, and a spare
|
|
44
|
+
// renderer is kept warm for the next navigation.
|
|
45
|
+
'--disable-features=BackForwardCache,SpareRendererForSitePerProcess,Translate,MediaRouter,OptimizationHints,InterestFeedContentSuggestions,AutofillServerCommunication',
|
|
46
|
+
];
|
|
47
|
+
|
|
48
|
+
function defaultLog(line) {
|
|
49
|
+
process.stderr.write(`${JSON.stringify(line)}\n`);
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
export async function serveTabs({
|
|
53
|
+
chromeBin,
|
|
54
|
+
chromeArgs = [],
|
|
55
|
+
headless = false,
|
|
56
|
+
proxyServer = null,
|
|
57
|
+
maxTabs = DEFAULT_MAX_TABS,
|
|
58
|
+
tabSeconds = DEFAULT_TAB_SECONDS,
|
|
59
|
+
sampleSeconds = DEFAULT_SAMPLE_SECONDS,
|
|
60
|
+
host = '127.0.0.1',
|
|
61
|
+
port = 0,
|
|
62
|
+
log = defaultLog,
|
|
63
|
+
}) {
|
|
64
|
+
if (!chromeBin) throw new Error('chromeBin is required');
|
|
65
|
+
if (!Number.isInteger(maxTabs) || maxTabs < 1) throw new Error('maxTabs must be a positive integer');
|
|
66
|
+
if (!(tabSeconds > 0)) throw new Error('tabSeconds must be positive');
|
|
67
|
+
|
|
68
|
+
const waiting = [];
|
|
69
|
+
const held = new Set();
|
|
70
|
+
let chrome = null;
|
|
71
|
+
let starting = null;
|
|
72
|
+
let closed = false;
|
|
73
|
+
|
|
74
|
+
async function launchChrome() {
|
|
75
|
+
const profile = mkdtempSync(join(tmpdir(), 'browser-tools-tabs-'));
|
|
76
|
+
const args = [...LEAN_CHROME_ARGS, `--user-data-dir=${profile}`, '--remote-debugging-address=127.0.0.1', '--remote-debugging-port=0'];
|
|
77
|
+
if (headless) args.push('--headless=new');
|
|
78
|
+
if (proxyServer) {
|
|
79
|
+
// Chrome sends loopback addresses around a proxy unless told otherwise, and WebRTC around it
|
|
80
|
+
// over UDP, either of which would let a page reach what the proxy refuses.
|
|
81
|
+
args.push(`--proxy-server=${proxyServer}`, '--proxy-bypass-list=<-loopback>', '--webrtc-ip-handling-policy=disable_non_proxied_udp');
|
|
82
|
+
}
|
|
83
|
+
args.push(...chromeArgs);
|
|
84
|
+
const child = spawn(chromeBin, args, { stdio: ['ignore', 'ignore', 'pipe'] });
|
|
85
|
+
let stderr = '';
|
|
86
|
+
child.stderr.on('data', (chunk) => {
|
|
87
|
+
stderr = (stderr + chunk).slice(-2000);
|
|
88
|
+
});
|
|
89
|
+
const exited = new Promise((resolve) => child.once('exit', (code, signal) => resolve({ code, signal })));
|
|
90
|
+
const portFile = join(profile, 'DevToolsActivePort');
|
|
91
|
+
const deadline = Date.now() + CHROME_START_TIMEOUT_MS;
|
|
92
|
+
let exit = null;
|
|
93
|
+
exited.then((value) => {
|
|
94
|
+
exit = value;
|
|
95
|
+
});
|
|
96
|
+
while (!existsSync(portFile) || readFileSync(portFile, 'utf8').split('\n').length < 2) {
|
|
97
|
+
if (exit) throw new Error(`Chrome exited while starting (${exit.signal ?? exit.code}): ${stderr.trim().slice(-500)}`);
|
|
98
|
+
if (Date.now() > deadline) {
|
|
99
|
+
child.kill('SIGKILL');
|
|
100
|
+
throw new Error(`Chrome did not start within ${CHROME_START_TIMEOUT_MS}ms`);
|
|
101
|
+
}
|
|
102
|
+
await new Promise((resolve) => setTimeout(resolve, 50));
|
|
103
|
+
}
|
|
104
|
+
const [cdpPort, path] = readFileSync(portFile, 'utf8').split('\n');
|
|
105
|
+
const cdpUrl = `ws://127.0.0.1:${cdpPort}${path}`;
|
|
106
|
+
const browser = await connectOwnBrowser(cdpUrl, { targetFilter: () => false });
|
|
107
|
+
const session = await browser.target().createCDPSession();
|
|
108
|
+
const started = { child, browser, session, cdpUrl, cdpPort: Number(cdpPort), profile, exited };
|
|
109
|
+
exited.then(({ code, signal }) => {
|
|
110
|
+
if (chrome === started) chrome = null;
|
|
111
|
+
rmSync(profile, { recursive: true, force: true });
|
|
112
|
+
if (!closed) log({ event: 'browser_exit', code, signal });
|
|
113
|
+
});
|
|
114
|
+
log({ event: 'browser_start', pid: child.pid });
|
|
115
|
+
return started;
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
async function ensureChrome() {
|
|
119
|
+
if (chrome && chrome.browser.connected && chrome.child.exitCode === null && chrome.child.signalCode === null) return chrome;
|
|
120
|
+
chrome = null;
|
|
121
|
+
starting ??= launchChrome().finally(() => {
|
|
122
|
+
starting = null;
|
|
123
|
+
});
|
|
124
|
+
chrome = await starting;
|
|
125
|
+
return chrome;
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
function pump() {
|
|
129
|
+
while (!closed && held.size < maxTabs && waiting.length > 0) {
|
|
130
|
+
const lease = waiting.shift();
|
|
131
|
+
held.add(lease);
|
|
132
|
+
void grant(lease);
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
function write(lease, line) {
|
|
137
|
+
if (!lease.response.writableEnded && !lease.response.destroyed) lease.response.write(`${JSON.stringify(line)}\n`);
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
function end(lease, ended) {
|
|
141
|
+
if (lease.ended) return;
|
|
142
|
+
lease.ended = ended;
|
|
143
|
+
clearTimeout(lease.timer);
|
|
144
|
+
held.delete(lease);
|
|
145
|
+
if (lease.context && lease.chrome) {
|
|
146
|
+
lease.chrome.session.send('Target.disposeBrowserContext', { browserContextId: lease.context }).catch(() => {});
|
|
147
|
+
}
|
|
148
|
+
log({
|
|
149
|
+
event: 'tab_lease',
|
|
150
|
+
client: lease.client,
|
|
151
|
+
waited_ms: (lease.granted ?? Date.now()) - lease.asked,
|
|
152
|
+
held_ms: lease.granted ? Date.now() - lease.granted : 0,
|
|
153
|
+
ended,
|
|
154
|
+
tabs: held.size,
|
|
155
|
+
waiting: waiting.length,
|
|
156
|
+
});
|
|
157
|
+
if (ended !== 'released') {
|
|
158
|
+
write(lease, ended === 'error' ? { error: lease.error } : { ended });
|
|
159
|
+
lease.response.end();
|
|
160
|
+
}
|
|
161
|
+
pump();
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
async function grant(lease) {
|
|
165
|
+
try {
|
|
166
|
+
const running = await ensureChrome();
|
|
167
|
+
lease.chrome = running;
|
|
168
|
+
const { browserContextId } = await running.session.send('Target.createBrowserContext', {
|
|
169
|
+
disposeOnDetach: false,
|
|
170
|
+
...(proxyServer ? { proxyServer, proxyBypassList: '<-loopback>' } : {}),
|
|
171
|
+
});
|
|
172
|
+
lease.context = browserContextId;
|
|
173
|
+
await running.session.send('Browser.setDownloadBehavior', { behavior: 'deny', browserContextId });
|
|
174
|
+
// The tab opens on an address of its own, which is how a client finds it among the browser's targets.
|
|
175
|
+
const tabUrl = `about:blank#${randomUUID()}`;
|
|
176
|
+
const { targetId } = await running.session.send('Target.createTarget', { url: tabUrl, browserContextId });
|
|
177
|
+
if (lease.gone) return end(lease, 'released');
|
|
178
|
+
lease.granted = Date.now();
|
|
179
|
+
lease.timer = setTimeout(() => end(lease, 'expired'), tabSeconds * 1000);
|
|
180
|
+
write(lease, { cdp_url: running.cdpUrl, target_id: targetId, tab_url: tabUrl, deadline: lease.granted + tabSeconds * 1000 });
|
|
181
|
+
} catch (error) {
|
|
182
|
+
lease.error = error instanceof Error ? error.message : String(error);
|
|
183
|
+
end(lease, 'error');
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
const server = createServer((request, response) => {
|
|
188
|
+
const url = new URL(request.url, 'http://broker');
|
|
189
|
+
if (request.method !== 'GET' || url.pathname !== '/lease') {
|
|
190
|
+
response.writeHead(404).end();
|
|
191
|
+
return;
|
|
192
|
+
}
|
|
193
|
+
if (closed) {
|
|
194
|
+
response.writeHead(503).end();
|
|
195
|
+
return;
|
|
196
|
+
}
|
|
197
|
+
const lease = { client: url.searchParams.get('client') || 'unnamed', asked: Date.now(), response, gone: false };
|
|
198
|
+
response.writeHead(200, { 'content-type': 'application/x-ndjson', 'cache-control': 'no-store' });
|
|
199
|
+
response.flushHeaders();
|
|
200
|
+
response.on('close', () => {
|
|
201
|
+
lease.gone = true;
|
|
202
|
+
const queued = waiting.indexOf(lease);
|
|
203
|
+
if (queued >= 0) waiting.splice(queued, 1);
|
|
204
|
+
else if (lease.granted) end(lease, 'released');
|
|
205
|
+
});
|
|
206
|
+
waiting.push(lease);
|
|
207
|
+
pump();
|
|
208
|
+
});
|
|
209
|
+
server.requestTimeout = 0;
|
|
210
|
+
await new Promise((resolve, reject) => {
|
|
211
|
+
server.once('error', reject);
|
|
212
|
+
server.listen(port, host, resolve);
|
|
213
|
+
});
|
|
214
|
+
|
|
215
|
+
const sampler =
|
|
216
|
+
process.platform === 'linux'
|
|
217
|
+
? setInterval(() => {
|
|
218
|
+
if (!chrome) return;
|
|
219
|
+
const sample = chromeMemory(chrome.child.pid);
|
|
220
|
+
if (sample) log({ event: 'browser_memory', ...sample, tabs: held.size, waiting: waiting.length });
|
|
221
|
+
}, sampleSeconds * 1000)
|
|
222
|
+
: null;
|
|
223
|
+
sampler?.unref();
|
|
224
|
+
|
|
225
|
+
const address = server.address();
|
|
226
|
+
return {
|
|
227
|
+
url: `http://${host}:${address.port}`,
|
|
228
|
+
get cdpPort() {
|
|
229
|
+
return chrome?.cdpPort ?? null;
|
|
230
|
+
},
|
|
231
|
+
async close() {
|
|
232
|
+
closed = true;
|
|
233
|
+
if (sampler) clearInterval(sampler);
|
|
234
|
+
for (const lease of [...held]) end(lease, 'closed');
|
|
235
|
+
for (const lease of waiting.splice(0)) lease.response.writeHead?.(503) && lease.response.end();
|
|
236
|
+
server.closeAllConnections();
|
|
237
|
+
await new Promise((resolve) => server.close(resolve));
|
|
238
|
+
const running = chrome ?? (await starting?.catch(() => null));
|
|
239
|
+
if (!running) return;
|
|
240
|
+
await running.session.send('Browser.close').catch(() => {});
|
|
241
|
+
running.browser.disconnect().catch?.(() => {});
|
|
242
|
+
const stopped = await Promise.race([
|
|
243
|
+
running.exited.then(() => true),
|
|
244
|
+
new Promise((resolve) => setTimeout(() => resolve(false), CHROME_STOP_TIMEOUT_MS)),
|
|
245
|
+
]);
|
|
246
|
+
if (!stopped) {
|
|
247
|
+
running.child.kill('SIGKILL');
|
|
248
|
+
await running.exited;
|
|
249
|
+
}
|
|
250
|
+
},
|
|
251
|
+
};
|
|
252
|
+
}
|
|
253
|
+
|
|
254
|
+
const PROCESS_TYPES = { renderer: 'renderer', 'gpu-process': 'gpu', utility: 'utility', zygote: 'zygote' };
|
|
255
|
+
|
|
256
|
+
/** Proportional set size of Chrome's process tree, by process type, from /proc. */
|
|
257
|
+
export function chromeMemory(rootPid) {
|
|
258
|
+
const children = new Map();
|
|
259
|
+
for (const name of readdirSync('/proc')) {
|
|
260
|
+
if (!/^\d+$/.test(name)) continue;
|
|
261
|
+
try {
|
|
262
|
+
const stat = readFileSync(`/proc/${name}/stat`, 'utf8');
|
|
263
|
+
const ppid = Number(stat.slice(stat.lastIndexOf(')') + 2).split(' ')[1]);
|
|
264
|
+
if (!children.has(ppid)) children.set(ppid, []);
|
|
265
|
+
children.get(ppid).push(Number(name));
|
|
266
|
+
} catch {
|
|
267
|
+
// The process exited while the table was read.
|
|
268
|
+
}
|
|
269
|
+
}
|
|
270
|
+
const pss = { browser: 0, renderer: 0, gpu: 0, utility: 0, zygote: 0, other: 0 };
|
|
271
|
+
let processes = 0;
|
|
272
|
+
let renderers = 0;
|
|
273
|
+
const pending = [rootPid];
|
|
274
|
+
while (pending.length > 0) {
|
|
275
|
+
const pid = pending.pop();
|
|
276
|
+
pending.push(...(children.get(pid) ?? []));
|
|
277
|
+
try {
|
|
278
|
+
const rollup = readFileSync(`/proc/${pid}/smaps_rollup`, 'utf8');
|
|
279
|
+
const kib = Number(/^Pss:\s+(\d+)/m.exec(rollup)?.[1] ?? 0);
|
|
280
|
+
const type = /--type=([\w-]+)/.exec(readFileSync(`/proc/${pid}/cmdline`, 'utf8').replaceAll('\0', ' '))?.[1];
|
|
281
|
+
const key = pid === rootPid ? 'browser' : (PROCESS_TYPES[type] ?? 'other');
|
|
282
|
+
pss[key] += kib;
|
|
283
|
+
processes += 1;
|
|
284
|
+
if (key === 'renderer') renderers += 1;
|
|
285
|
+
} catch {
|
|
286
|
+
// The process exited while it was read.
|
|
287
|
+
}
|
|
288
|
+
}
|
|
289
|
+
if (processes === 0) return null;
|
|
290
|
+
const mib = (kib) => Math.round(kib / 1024);
|
|
291
|
+
return {
|
|
292
|
+
total_mib: mib(Object.values(pss).reduce((sum, kib) => sum + kib, 0)),
|
|
293
|
+
pss_mib: Object.fromEntries(Object.entries(pss).map(([key, kib]) => [key, mib(kib)])),
|
|
294
|
+
processes,
|
|
295
|
+
renderers,
|
|
296
|
+
};
|
|
297
|
+
}
|
|
298
|
+
|
|
299
|
+
/**
|
|
300
|
+
* Waits for a tab from the broker at `brokerUrl`. The tab is the client's until `release()` is called
|
|
301
|
+
* or the process exits; `ended` settles with 'expired' if the broker took the tab back first.
|
|
302
|
+
* The tab is the page at `tabUrl` in the browser at `cdpUrl`.
|
|
303
|
+
*/
|
|
304
|
+
export function leaseTab(brokerUrl, { client = 'unnamed' } = {}) {
|
|
305
|
+
return new Promise((resolve, reject) => {
|
|
306
|
+
let granted = null;
|
|
307
|
+
let finish;
|
|
308
|
+
const ended = new Promise((resolveEnded) => {
|
|
309
|
+
finish = resolveEnded;
|
|
310
|
+
});
|
|
311
|
+
const request = httpGet(`${brokerUrl}/lease?client=${encodeURIComponent(client)}`, { agent: false }, (response) => {
|
|
312
|
+
if (response.statusCode !== 200) {
|
|
313
|
+
response.resume();
|
|
314
|
+
reject(new Error(`the tab broker refused the lease with HTTP ${response.statusCode}`));
|
|
315
|
+
return;
|
|
316
|
+
}
|
|
317
|
+
response.setEncoding('utf8');
|
|
318
|
+
let buffer = '';
|
|
319
|
+
response.on('data', (chunk) => {
|
|
320
|
+
buffer += chunk;
|
|
321
|
+
for (let index = buffer.indexOf('\n'); index >= 0; index = buffer.indexOf('\n')) {
|
|
322
|
+
const line = JSON.parse(buffer.slice(0, index));
|
|
323
|
+
buffer = buffer.slice(index + 1);
|
|
324
|
+
if (line.error) {
|
|
325
|
+
reject(new Error(`the tab broker could not open a tab: ${line.error}`));
|
|
326
|
+
} else if (!granted) {
|
|
327
|
+
granted = {
|
|
328
|
+
cdpUrl: line.cdp_url,
|
|
329
|
+
targetId: line.target_id,
|
|
330
|
+
tabUrl: line.tab_url,
|
|
331
|
+
deadline: line.deadline,
|
|
332
|
+
ended,
|
|
333
|
+
release: () => request.destroy(),
|
|
334
|
+
};
|
|
335
|
+
resolve(granted);
|
|
336
|
+
} else if (line.ended) {
|
|
337
|
+
finish(line.ended);
|
|
338
|
+
}
|
|
339
|
+
}
|
|
340
|
+
});
|
|
341
|
+
response.on('close', () => {
|
|
342
|
+
if (!granted) reject(new Error('the tab broker closed the lease before granting a tab'));
|
|
343
|
+
finish('released');
|
|
344
|
+
});
|
|
345
|
+
});
|
|
346
|
+
request.on('error', (error) => {
|
|
347
|
+
if (!granted) reject(error);
|
|
348
|
+
finish('released');
|
|
349
|
+
});
|
|
350
|
+
});
|
|
351
|
+
}
|