dsh-inner-browser-use 0.46.0 → 0.47.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -0
- package/README.zh.md +7 -0
- package/docs/BENCH.md +51 -0
- package/docs/MATURITY.md +52 -0
- package/docs/RESOURCES.md +18 -9
- package/docs/VERIFY.md +18 -0
- package/docs/measure-competitors-2026-11.json +523 -0
- package/package.json +4 -2
- package/scripts/bench-resource.mjs +173 -0
- package/sidecar.js +49 -1
package/README.md
CHANGED
|
@@ -78,6 +78,13 @@ sign-in here is still there the next day.
|
|
|
78
78
|
- **Shutdown.** The browser is asked to close rather than killed, which is when Chrome writes
|
|
79
79
|
the last of its cookies.
|
|
80
80
|
|
|
81
|
+
## What it costs
|
|
82
|
+
|
|
83
|
+
Measured on the same machine with the same harness as Playwright MCP and Chrome
|
|
84
|
+
DevTools MCP: idle RSS **551 MB** (1414 / 1280), idle CPU **0.03–0.32%**
|
|
85
|
+
(38.1% / 36.3%), six tabs in four renderer processes, and a wake in
|
|
86
|
+
**0.84–0.86 s** instead of a 4.6–5.1 s restart. `npm run bench` reproduces it.
|
|
87
|
+
|
|
81
88
|
## When nobody is looking
|
|
82
89
|
|
|
83
90
|
A browser used every two seconds rests after two minutes; one used every ninety seconds waits
|
package/README.zh.md
CHANGED
|
@@ -70,6 +70,13 @@ profile 是独立目录:`~/.dsh/inner-browser-use/chrome/default`。它在产
|
|
|
70
70
|
- **Harness 重启。** 重新打开同一个目录,cookie、`localStorage` 和 Chrome 存下的密码都留着。
|
|
71
71
|
- **退出。** 关浏览器是请它自己关(`Browser.close`),Chrome 在这一步才把 cookie 落盘。
|
|
72
72
|
|
|
73
|
+
## 常驻成本
|
|
74
|
+
|
|
75
|
+
同机、同量法(Playwright MCP 与 Chrome DevTools MCP 用同一套 harness):空闲 RSS
|
|
76
|
+
**551 MB**(对方 1414 / 1280)、空闲 CPU **0.03–0.32%**(38.1% / 36.3%)、6 个页签共用
|
|
77
|
+
4 个渲染进程;休息后唤醒 **0.84–0.86 s** 回到原页,对方要整棵重启 4.6–5.1 s。
|
|
78
|
+
`npm run bench` 可复现,对照明细在 `docs/BENCH.md`。
|
|
79
|
+
|
|
73
80
|
## 没人看的时候
|
|
74
81
|
|
|
75
82
|
每两秒用一次的人,两分钟后就休息;每九十秒用一次的人能等九分钟——窗口取自这个人最近 64 次
|
package/docs/BENCH.md
CHANGED
|
@@ -1031,6 +1031,57 @@ not.
|
|
|
1031
1031
|
| mirror SSE | 15 fps(66 ms 一轮),4 s 收 60 帧,均值 97.8 KB |
|
|
1032
1032
|
| switch cost | 一次浏览器重启(`browser_open({flash:true})` → `browser_open({flash:false})`) |
|
|
1033
1033
|
|
|
1034
|
+
## Phase 25 — 和竞品同机同法的资源对照,0.47.0
|
|
1035
|
+
|
|
1036
|
+
竞品数字不是估的:`@playwright/mcp` 0.0.84 与 `chrome-devtools-mcp` 1.12.0 用同一个
|
|
1037
|
+
`/tmp/compbench/bench.mjs`、同一台机器(macOS 27.2 / arm64 / Node 24.16 / Chrome
|
|
1038
|
+
156.0.8078.80,无头、`--use-mock-keychain`)、同一目标页(example.com),每项重复取
|
|
1039
|
+
min/median/max。原始记录(含每进程 RSS、失败点、方法说明)在
|
|
1040
|
+
`docs/measure-competitors-2026-11.json`,可直接对账。
|
|
1041
|
+
|
|
1042
|
+
本插件一侧由 `scripts/bench-resource.mjs`(`npm run bench`)产出,同一套量法:真实进程树
|
|
1043
|
+
RSS、`ps -o time` 累计 CPU、同一页的读写往返。本地页面 /page(40 段正文+60 个链接)。
|
|
1044
|
+
|
|
1045
|
+
| what | this plugin 0.47.0 | Playwright MCP 0.0.84 | Chrome DevTools MCP 1.12.0 |
|
|
1046
|
+
| --- | --- | --- | --- |
|
|
1047
|
+
| 冷启动到可驱动页 | **1703–1757 ms**(机器忙时 2411) | 2464 / 2755 / 2887 | 1596 / 2185 / 2337 |
|
|
1048
|
+
| 读一页 | **2–3 ms**,2174 字 | 工具往返 24 ms(示例页) | 工具往返 15 ms |
|
|
1049
|
+
| 点/执行一次 | **0–1 ms** | 未单独计时 | 未单独计时 |
|
|
1050
|
+
| 空闲 RSS(整棵进程树) | **551 MB** / 8 进程 | 1414 MB / 9 进程(Chrome 1290) | 1280 MB / 8 进程(Chrome 1109) |
|
|
1051
|
+
| 空闲 CPU(单核占比) | **0.03–0.32%**(30 s 均值) | 38.1%(0–150 s 均值) | 36.3% |
|
|
1052
|
+
| 冻结(有人看、没人用) | CPU 0%,RSS 511–550 MB,唤醒 5–15 ms | 无此状态 | 无此状态 |
|
|
1053
|
+
| 休息(没人看) | **RSS 0**,唤醒 838–860 ms 回原页原地址 | 整棵关掉后冷启动 5133 ms | 无空闲回收 |
|
|
1054
|
+
| 页签切换 | 一个 `browser_tabs` 调用 | 3416 ms(其中 3392 是打开 wikipedia) | 2918 ms(2903 是打开) |
|
|
1055
|
+
|
|
1056
|
+
读法:
|
|
1057
|
+
|
|
1058
|
+
- 冷启动和 Chrome DevTools MCP 同一量级(中位 1703 vs 2185 ms),比 Playwright MCP 快一
|
|
1059
|
+
档;它最低一档 1596 ms 仍比我们的中位低,这一项不能算全面领先。
|
|
1060
|
+
- 空闲 CPU 差约两个数量级,来源是它默认不做任何节流,而我们把后台网络、组件更新、崩溃
|
|
1061
|
+
上报、翻译/媒体路由全关掉,再把页面冻结。
|
|
1062
|
+
- 空闲内存大约是一半:`--renderer-process-limit=4` 让 6 个页签共享 4 个渲染进程。
|
|
1063
|
+
- 休息后的唤醒 838–860 ms,对方恢复要整棵重启 4.6–5.1 s;但 Playwright MCP 确实有
|
|
1064
|
+
`--idle-timeout`(无头默认一小时),只是不区分有没有人在看、也不先冻结再关。这一条是
|
|
1065
|
+
「更细」而不是「只有我们有」。
|
|
1066
|
+
|
|
1067
|
+
### 这一版为数字做的改动
|
|
1068
|
+
|
|
1069
|
+
| change | measured effect |
|
|
1070
|
+
| --- | --- |
|
|
1071
|
+
| `--renderer-process-limit=4` | 空闲 RSS 660 → 551 MB;冷启动 2186 → 1703 ms |
|
|
1072
|
+
| 关掉后台网络 / 组件更新 / 崩溃上报 / 扩展 / 默认应用,功能表减去 5 项 | 空闲 CPU 1.87% → 0.03–0.32% |
|
|
1073
|
+
| `shed()`:冻结前 `gc()` + `Memory.simulatePressureNotification('critical')` | 冻结后 CPU 恒为 0;RSS 在有的轮次 812 → 511 MB,有的轮次不掉——掉多少取决于页面攒了多少缓存,不承诺固定回收量 |
|
|
1074
|
+
| 复用路径统一 `thaw()` | 冻结页被 `tabFor` / `openTab` / 并发合并任一路径拿到时都会先醒,不再当成活页读 |
|
|
1075
|
+
| 与 `--disable-gpu`、`--process-per-site`、`--renderer-process-limit=2/3` 对比 | gpu 关掉内存反升到 737 MB、空闲 CPU 8%;process-per-site 686 MB;limit=2 577 MB;limit=3 548 MB;limit=4 **540–551 MB** 且冷启动最好 |
|
|
1076
|
+
|
|
1077
|
+
### 方法上的边界
|
|
1078
|
+
|
|
1079
|
+
- 竞品的 RSS 是逐进程相加,共享页会重复计一次;我们同一量法,所以对比成立,绝对值偏高。
|
|
1080
|
+
- 竞品的空闲 CPU 高,一部分是 Chrome 自己在后台干活;用 `sleep 8` 走同一量法得 0.00
|
|
1081
|
+
CPU-s、再用 `top -l 2` 交叉验证过,不是量法造成的。
|
|
1082
|
+
- 页签切换那两行主要花在 wikipedia 加载上(同一工具换 example.com 是 65 / 151 ms),
|
|
1083
|
+
不是工具开销,别拿去当工具性能比较。
|
|
1084
|
+
|
|
1034
1085
|
## Phase 24 — 长期运行的资源回收,0.46.0
|
|
1035
1086
|
|
|
1036
1087
|
一台窗口隐藏的 Chrome 空转一晚就是几百 MB。这一版把「多久收」交给用法自己决定:记住这个人
|
package/docs/MATURITY.md
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# 成熟度(0.47.0,2026-11)
|
|
2
|
+
|
|
3
|
+
结论:**可日常用,边界清楚**。能力面已经越过「能开网页」的层次,长跑与资源回收这条链路是本版
|
|
4
|
+
重点并且有实测;短板集中在跨平台验证(只有 macOS 真机跑过)和少数只有真机才能验的交互(Flash、
|
|
5
|
+
镜像上的手感)。
|
|
6
|
+
|
|
7
|
+
## 交付面
|
|
8
|
+
|
|
9
|
+
| | |
|
|
10
|
+
| --- | --- |
|
|
11
|
+
| 版本 / 许可 | 0.47.0 / MIT |
|
|
12
|
+
| 仓库 | github.com/regrevia/dsh-inner-browser-use(main 与远端一致) |
|
|
13
|
+
| npm | `dsh-inner-browser-use@0.47.0` |
|
|
14
|
+
| 代码 | 约 11.9k 行(index 956 · sidecar 3683 · tools 4111 · read 2039 · rest 124 · client 954) |
|
|
15
|
+
| 工具 | 39 个 `browser_*`,模型侧零配置 |
|
|
16
|
+
| 文档 | README ×2、ARCHITECTURE、VERIFY(10 节)、BENCH(24 个阶段)、RESOURCES、ARTIFACTS、COMPATIBILITY、FLASH、ROADMAP |
|
|
17
|
+
| 验证脚本 | 27 个 `scripts/`,含 live / live-safety / breadth / soak / rest / doctor |
|
|
18
|
+
|
|
19
|
+
## 证据链
|
|
20
|
+
|
|
21
|
+
| 项 | 结果 |
|
|
22
|
+
| --- | --- |
|
|
23
|
+
| 单元测试 | `node --test test/*.test.mjs` → **305 项,305 通过,0 失败** |
|
|
24
|
+
| 资源对照 | 同机同法:冷启动中位 1703 ms、空闲 RSS 551 MB、空闲 CPU 0.03–0.32%、休息后 838–860 ms 唤醒;竞品 Playwright MCP 2755 ms / 1414 MB / 38.1%、Chrome DevTools MCP 2185 ms / 1280 MB / 36.3%(`docs/BENCH.md`、`docs/measure-competitors-2026-11.json`) |
|
|
25
|
+
| 长跑 | soak 24 轮 + 3 次真实休息:40.3 s,内存 **1183 → 912 MB**,页数始终 1,24 次读全部有答案 |
|
|
26
|
+
| 休息/唤醒 | `npm run rest`:休息后内存 0、状态说 `stopped`,再读自愈回原页,唤醒 838–860 ms |
|
|
27
|
+
| 资源策略 | p95 学习窗口(90 s–15 min / 20 s–2 min)、内存预算 2400 MB、页签 LRU 10 min、崩溃循环守卫 |
|
|
28
|
+
| 串行化 | 关闭单飞 + 启动等待;旧版「休息与调用相撞」的 `CDP session is closed` 已复现并修复,soak 不再失败 |
|
|
29
|
+
| 不变量 | 一个 chat 一页;两点钟不会被别人抢;右栏只有本会话能开;镜像外会话一律 403 |
|
|
30
|
+
|
|
31
|
+
## 已经比同类强的地方
|
|
32
|
+
|
|
33
|
+
- **冻结而不是关闭**:有人看但没人用时停驻页面(DOM/滚动/登录态保留),不是一刀切关掉
|
|
34
|
+
- **查看器感知**:没人看才真释放;有人看最多停驻
|
|
35
|
+
- **时间自己学**:不是拍一个固定超时,是这个人用法的 95 分位
|
|
36
|
+
- **内存当策略**:超预算先收 LRU 页签、降码率,再决定休息
|
|
37
|
+
- **唤醒回原页**:0.84–0.86 s(竞品恢复是整棵重启 4.6–5.1 s),接着原地址,不是冷启动白页
|
|
38
|
+
- **常驻成本**:6 个页签共用 4 个渲染进程,空闲 RSS 551 MB、空闲 CPU 0.03–0.32%;按同机同法测,竞品是 1280–1414 MB 和 36–38%
|
|
39
|
+
- 竞品对照与出处见 `docs/RESOURCES.md` 与 `docs/BENCH.md`(Playwright MCP 有 `--idle-timeout` 但只按时辰、Chrome DevTools MCP 没有、browser-use 只有 `keep_alive`、Stagehand 反而关掉 Chrome 节流)
|
|
40
|
+
|
|
41
|
+
## 短板(如实)
|
|
42
|
+
|
|
43
|
+
1. **跨平台**:只在 macOS + Chrome 上跑过;Windows/Linux 的进程树回收、`taskkill` 分支有代码但没实机验证(`COMPATIBILITY.md`)。
|
|
44
|
+
2. **真机交互**:Flash 分支需要本机 Electron 运行时可验证;镜像上的拖拽手感受帧率影响,只有 `adb`/人眼类检查,没有自动化回归。
|
|
45
|
+
3. **多 agent 隔离**:一个进程一台 Chrome、按 chat 分页;没有「每个 agent 独立浏览器」的强隔离档位(竞品同样没有做得更好的)。
|
|
46
|
+
4. **内存预算**:是策略而非硬限制;极端站点仍可能超过预算后再回收,不是提前拒绝。
|
|
47
|
+
5. **市场收录**:插件市场条目来自每日构建的 catalog,发版后自动跟上,无需 PR;本条目不代表示例/评分。
|
|
48
|
+
|
|
49
|
+
## 结论
|
|
50
|
+
|
|
51
|
+
从「能不能用」到「长期开着放不放心」这条线上,本版补齐了最后一段:没人用就真的不占资源,有人用就
|
|
52
|
+
立刻回来。剩下的风险主要是平台覆盖面与真机交互回归,不在核心链路。
|
package/docs/RESOURCES.md
CHANGED
|
@@ -64,15 +64,24 @@ person is looking at.
|
|
|
64
64
|
## Neighbours
|
|
65
65
|
|
|
66
66
|
Compared with the projects this one is usually put next to (`--idle-timeout`,
|
|
67
|
-
`keep_alive`, and their documentation)
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
|
72
|
-
|
|
|
73
|
-
|
|
|
74
|
-
|
|
|
75
|
-
|
|
|
67
|
+
`keep_alive`, and their documentation). The numbers come from the same harness
|
|
68
|
+
on the same machine; the raw record is
|
|
69
|
+
`docs/measure-competitors-2026-11.json` and the method is in `docs/BENCH.md`.
|
|
70
|
+
|
|
71
|
+
| project | idle handling | idle CPU / RSS | what a wake costs | notes |
|
|
72
|
+
| --- | --- | --- | --- | --- |
|
|
73
|
+
| Playwright MCP 0.0.84 | `--idle-timeout`, default one hour for headless, whole browser closed | 38.1% / 1414 MB | cold start, 5.1 s | does release, but never looks at whether anyone is watching and never parks a page first; each client gets its own Chrome |
|
|
74
|
+
| Chrome DevTools MCP 1.12.0 | none — the browser lives until the process exits | 36.3% / 1280 MB | n/a | one dispose chain, no idle clock, screencast never pauses for a detached viewer |
|
|
75
|
+
| browser-use | `keep_alive` flag, no timeout value | not measured | n/a | teardown skipped while `keep_alive`; no idle timer, no memory policy |
|
|
76
|
+
| Stagehand | none | not measured | n/a | deliberately disables Chrome's background throttling, then tears down and removes the profile |
|
|
77
|
+
| this plugin 0.47.0 | viewer-aware ladder, timings learned from use, memory pressure, single-flight close | 0.03–0.32% / 551 MB | 0.84–0.86 s, same page and address | freeze keeps the DOM; rest keeps the cookies; nothing cold-starts unless the process is new |
|
|
78
|
+
|
|
79
|
+
Playwright MCP's idle timeout was measured working (`--idle-timeout=6000` left no
|
|
80
|
+
Chrome after 14 s idle while the MCP process stayed up), so "we are the only one
|
|
81
|
+
that releases" would be false. What differs is the shape of the ladder: it is
|
|
82
|
+
one timer and one action, while this plugin never closes a page someone is
|
|
83
|
+
watching, freezes before it closes, and derives both windows from how often the
|
|
84
|
+
person actually calls.
|
|
76
85
|
|
|
77
86
|
The gaps that remain in the field, and that this document treats as the design
|
|
78
87
|
brief: nobody freezes a page instead of closing it; nobody pauses a screencast
|
package/docs/VERIFY.md
CHANGED
|
@@ -234,3 +234,21 @@ navigation — as a session that is not the owner. Both must answer 403, and the
|
|
|
234
234
|
owner's stream must answer 200. Three `true` lines is the bar; `open · owner
|
|
235
235
|
none` as the first line only means nothing is on screen, and the two refusals
|
|
236
236
|
still have to hold.
|
|
237
|
+
|
|
238
|
+
## 11. Compare against the neighbours
|
|
239
|
+
|
|
240
|
+
```bash
|
|
241
|
+
npm run bench # writes JSON to stdout, exit 1 if a check fails
|
|
242
|
+
npm run bench -- --json /tmp/bench.json
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
Cold start, one read, one action, 30 s idle, freeze, rest, wake — each with the
|
|
246
|
+
real process tree's RSS and CPU seconds. Add flags for an A/B without editing
|
|
247
|
+
the source:
|
|
248
|
+
|
|
249
|
+
```bash
|
|
250
|
+
DSH_INNER_BROWSER_CHROME_FLAGS="--renderer-process-limit=2" npm run bench
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
The competitor side of the comparison and its raw record live in
|
|
254
|
+
`docs/BENCH.md` and `docs/measure-competitors-2026-11.json`.
|
|
@@ -0,0 +1,523 @@
|
|
|
1
|
+
{
|
|
2
|
+
"generated_at": "2026-11-01T17:15:17.022Z",
|
|
3
|
+
"machine": {
|
|
4
|
+
"arch": "arm64",
|
|
5
|
+
"os": "macOS 27.2",
|
|
6
|
+
"os_build": "26B123",
|
|
7
|
+
"node": "v24.16.0",
|
|
8
|
+
"npm": "11.13.0",
|
|
9
|
+
"chrome": "156.0.8078.80 (Google Chrome.app, arm64)",
|
|
10
|
+
"note": "No other browser was running during the measurements (see method.failure_points)."
|
|
11
|
+
},
|
|
12
|
+
"competitors": {
|
|
13
|
+
"playwright_mcp": {
|
|
14
|
+
"package": "@playwright/mcp",
|
|
15
|
+
"package_version": "0.0.84",
|
|
16
|
+
"launch_command": "npx -y @playwright/mcp@latest --headless --isolated --config /tmp/compbench/pw-config.json",
|
|
17
|
+
"mcp_server_info": {
|
|
18
|
+
"name": "Playwright",
|
|
19
|
+
"version": "1.65.0-alpha-2026-10-30"
|
|
20
|
+
},
|
|
21
|
+
"cold_start_ms": {
|
|
22
|
+
"runs_ms": [
|
|
23
|
+
2887,
|
|
24
|
+
2755,
|
|
25
|
+
2464
|
|
26
|
+
],
|
|
27
|
+
"min": 2464,
|
|
28
|
+
"median": 2755,
|
|
29
|
+
"max": 2887,
|
|
30
|
+
"definition": "wall clock from spawn(npx) to the navigate tool call returning the page URL, npx package cache warm (already downloaded). For chrome-devtools-mcp this includes the mandatory list_pages call that the tool schema requires to obtain a pageId; that extra round trip is inherent to its API."
|
|
31
|
+
},
|
|
32
|
+
"first_npx_ms": 3439,
|
|
33
|
+
"first_npx_ms_method": "wall clock of the very first `npx -y <pkg>@latest --help` on a cold npm cache (download + install + start); later runs reuse the npx cache",
|
|
34
|
+
"idle_rss_mb": {
|
|
35
|
+
"total_mb": 1414,
|
|
36
|
+
"chrome_tree_mb": 1290.1,
|
|
37
|
+
"node_npx_mb": 123.9,
|
|
38
|
+
"chrome_process_count": 8,
|
|
39
|
+
"total_process_count": 10,
|
|
40
|
+
"chrome_processes": [
|
|
41
|
+
{
|
|
42
|
+
"pid": 27560,
|
|
43
|
+
"rss_mb": 581.3
|
|
44
|
+
},
|
|
45
|
+
{
|
|
46
|
+
"pid": 29419,
|
|
47
|
+
"rss_mb": 134.8
|
|
48
|
+
},
|
|
49
|
+
{
|
|
50
|
+
"pid": 27575,
|
|
51
|
+
"rss_mb": 125.2
|
|
52
|
+
},
|
|
53
|
+
{
|
|
54
|
+
"pid": 27572,
|
|
55
|
+
"rss_mb": 121.7
|
|
56
|
+
},
|
|
57
|
+
{
|
|
58
|
+
"pid": 27566,
|
|
59
|
+
"rss_mb": 101.5
|
|
60
|
+
},
|
|
61
|
+
{
|
|
62
|
+
"pid": 27567,
|
|
63
|
+
"rss_mb": 82
|
|
64
|
+
},
|
|
65
|
+
{
|
|
66
|
+
"pid": 27574,
|
|
67
|
+
"rss_mb": 78.8
|
|
68
|
+
},
|
|
69
|
+
{
|
|
70
|
+
"pid": 27568,
|
|
71
|
+
"rss_mb": 65.4
|
|
72
|
+
}
|
|
73
|
+
],
|
|
74
|
+
"method": "sum of `ps -Ao rss=` (KB) over every descendant of the spawned npx pid, 180 s after navigate returned; chrome_tree_mb restricts that sum to the subtree rooted at the launched Google Chrome binary. macOS RSS counts private + shared mapped pages per process, so summing over Chrome helpers (each mapping the same Chrome Framework) overstates true memory use; the per-process list is included so the double counting is visible. No dedup is applied because the task asks for the RSS sum."
|
|
75
|
+
},
|
|
76
|
+
"restart_ms": 5133,
|
|
77
|
+
"tab_switch_ms": {
|
|
78
|
+
"total_ms": 3416,
|
|
79
|
+
"open_second_page_ms": 3392,
|
|
80
|
+
"switch_back_and_read_ms": 24,
|
|
81
|
+
"second_page_url": "https://en.wikipedia.org/wiki/Main_Page",
|
|
82
|
+
"second_page_loaded": true,
|
|
83
|
+
"read_ok": true,
|
|
84
|
+
"control_open_second_page_same_tool_example_com_ms": 65
|
|
85
|
+
},
|
|
86
|
+
"idle_cpu_pct": {
|
|
87
|
+
"avg_pct_all_windows": 38.07,
|
|
88
|
+
"steady_state_avg_pct_after_60s": 40.38,
|
|
89
|
+
"peak_window_pct": 62.87,
|
|
90
|
+
"windows": [
|
|
91
|
+
{
|
|
92
|
+
"window": "0-30s after navigate",
|
|
93
|
+
"avg_cpu_pct_of_one_core": 6.37,
|
|
94
|
+
"pids": 9,
|
|
95
|
+
"top": [
|
|
96
|
+
{
|
|
97
|
+
"pid": 33217,
|
|
98
|
+
"sec": 1.77
|
|
99
|
+
},
|
|
100
|
+
{
|
|
101
|
+
"pid": 33223,
|
|
102
|
+
"sec": 0.04
|
|
103
|
+
},
|
|
104
|
+
{
|
|
105
|
+
"pid": 33229,
|
|
106
|
+
"sec": 0.04
|
|
107
|
+
}
|
|
108
|
+
]
|
|
109
|
+
},
|
|
110
|
+
{
|
|
111
|
+
"window": "30-60s after navigate",
|
|
112
|
+
"avg_cpu_pct_of_one_core": 62.87,
|
|
113
|
+
"pids": 9,
|
|
114
|
+
"top": [
|
|
115
|
+
{
|
|
116
|
+
"pid": 33217,
|
|
117
|
+
"sec": 18.85
|
|
118
|
+
},
|
|
119
|
+
{
|
|
120
|
+
"pid": 33224,
|
|
121
|
+
"sec": 0.01
|
|
122
|
+
}
|
|
123
|
+
]
|
|
124
|
+
},
|
|
125
|
+
{
|
|
126
|
+
"window": "60-90s after navigate",
|
|
127
|
+
"avg_cpu_pct_of_one_core": 7.13,
|
|
128
|
+
"pids": 9,
|
|
129
|
+
"top": [
|
|
130
|
+
{
|
|
131
|
+
"pid": 33217,
|
|
132
|
+
"sec": 2.09
|
|
133
|
+
},
|
|
134
|
+
{
|
|
135
|
+
"pid": 33216,
|
|
136
|
+
"sec": 0.01
|
|
137
|
+
},
|
|
138
|
+
{
|
|
139
|
+
"pid": 33223,
|
|
140
|
+
"sec": 0.01
|
|
141
|
+
}
|
|
142
|
+
]
|
|
143
|
+
},
|
|
144
|
+
{
|
|
145
|
+
"window": "90-120s after navigate",
|
|
146
|
+
"avg_cpu_pct_of_one_core": 52.6,
|
|
147
|
+
"pids": 9,
|
|
148
|
+
"top": [
|
|
149
|
+
{
|
|
150
|
+
"pid": 33217,
|
|
151
|
+
"sec": 15.67
|
|
152
|
+
},
|
|
153
|
+
{
|
|
154
|
+
"pid": 33199,
|
|
155
|
+
"sec": 0.06
|
|
156
|
+
},
|
|
157
|
+
{
|
|
158
|
+
"pid": 33216,
|
|
159
|
+
"sec": 0.05
|
|
160
|
+
}
|
|
161
|
+
]
|
|
162
|
+
},
|
|
163
|
+
{
|
|
164
|
+
"window": "120-150s after navigate",
|
|
165
|
+
"avg_cpu_pct_of_one_core": 61.4,
|
|
166
|
+
"pids": 9,
|
|
167
|
+
"top": [
|
|
168
|
+
{
|
|
169
|
+
"pid": 33217,
|
|
170
|
+
"sec": 18.42
|
|
171
|
+
}
|
|
172
|
+
]
|
|
173
|
+
}
|
|
174
|
+
],
|
|
175
|
+
"cross_check_bench_run_avg_pct": 35.75,
|
|
176
|
+
"cross_check_top_instantaneous_pct": [
|
|
177
|
+
{
|
|
178
|
+
"window": "0-15s",
|
|
179
|
+
"delta_method_pct": 28.7,
|
|
180
|
+
"top_l2_pct": 0
|
|
181
|
+
},
|
|
182
|
+
{
|
|
183
|
+
"window": "15-30s",
|
|
184
|
+
"delta_method_pct": 59.6,
|
|
185
|
+
"top_l2_pct": 49.6
|
|
186
|
+
},
|
|
187
|
+
{
|
|
188
|
+
"window": "30-45s",
|
|
189
|
+
"delta_method_pct": 26.5,
|
|
190
|
+
"top_l2_pct": 45.1
|
|
191
|
+
},
|
|
192
|
+
{
|
|
193
|
+
"window": "45-60s",
|
|
194
|
+
"delta_method_pct": 0.2,
|
|
195
|
+
"top_l2_pct": 0
|
|
196
|
+
},
|
|
197
|
+
{
|
|
198
|
+
"window": "60-75s",
|
|
199
|
+
"delta_method_pct": 64,
|
|
200
|
+
"top_l2_pct": 61.4
|
|
201
|
+
},
|
|
202
|
+
{
|
|
203
|
+
"window": "75-90s",
|
|
204
|
+
"delta_method_pct": 50.6,
|
|
205
|
+
"top_l2_pct": 48.3
|
|
206
|
+
},
|
|
207
|
+
{
|
|
208
|
+
"window": "90-105s",
|
|
209
|
+
"delta_method_pct": 29.7,
|
|
210
|
+
"top_l2_pct": 0
|
|
211
|
+
},
|
|
212
|
+
{
|
|
213
|
+
"window": "105-120s",
|
|
214
|
+
"delta_method_pct": 86.7,
|
|
215
|
+
"top_l2_pct": 53.4
|
|
216
|
+
}
|
|
217
|
+
],
|
|
218
|
+
"method_controls": {
|
|
219
|
+
"sleep_process_cpu_seconds": 0,
|
|
220
|
+
"note": "The same ps-time delta applied to a plain `sleep 8` yields 0.00 CPU s, and top -l 2 (instantaneous per-process %CPU) on the Chrome main process tracks the delta method within sampling noise, so the numbers are CPU actually burned by the tree and not an accounting artifact."
|
|
221
|
+
},
|
|
222
|
+
"error": null,
|
|
223
|
+
"method": "Instantaneous CPU of the benchmark process tree, not the ps %cpu column. Every 30 s the cumulative CPU seconds of all tree pids are read from `ps -Ao pid=,time=`, and the delta divided by the 30 s wall window gives the average fraction of one core used by the whole tree (npx + node + all Chrome processes). Each window starts where the previous ended, from navigate +0 s; the early windows include Chrome post-load work and the ones after 60 s are the settled state. The ps %cpu column is a lifetime average and inflates for short-lived helpers, so it is only kept for reference."
|
|
224
|
+
},
|
|
225
|
+
"auto_release": {
|
|
226
|
+
"has_mechanism": true,
|
|
227
|
+
"parameter": "--idle-timeout <ms> | env PLAYWRIGHT_MCP_IDLE_TIMEOUT | config browser.idle",
|
|
228
|
+
"default_behavior": "one hour for headless browsers it launched, no timeout for headed or attached ones, 0 disables; the next tool call relaunches the browser",
|
|
229
|
+
"evidence": [
|
|
230
|
+
"https://github.com/microsoft/playwright-mcp/blob/main/README.md (README.md:431 in @playwright/mcp@0.0.84): \"--idle-timeout <timeout> | close the browser after this many milliseconds without a completed tool call, the next tool call relaunches it.\"",
|
|
231
|
+
"config.d.ts:240 in @playwright/mcp@0.0.84: \"idle?: number;\" with the comment \"Close the browser after this many milliseconds without a tool call, and relaunch it on the next one.\""
|
|
232
|
+
],
|
|
233
|
+
"empirical": {
|
|
234
|
+
"flag_used": "--idle-timeout=6000",
|
|
235
|
+
"browser_auto_closed": true,
|
|
236
|
+
"details": "navigate, then 14 s without any tool call: chrome roots alive = 0, MCP process alive = true"
|
|
237
|
+
}
|
|
238
|
+
},
|
|
239
|
+
"notes": [
|
|
240
|
+
"cold_start run1 2887ms",
|
|
241
|
+
"cpu interval 1: 65.11% (ps %cpu sum for reference 63.5%) pids=9",
|
|
242
|
+
"cpu interval 2: 62.38% (ps %cpu sum for reference 63.7%) pids=9",
|
|
243
|
+
"cpu interval 3: 57.41% (ps %cpu sum for reference 13.1%) pids=9",
|
|
244
|
+
"cpu interval 4: 0.4% (ps %cpu sum for reference 0%) pids=9",
|
|
245
|
+
"cpu interval 5: 0.2% (ps %cpu sum for reference 0%) pids=9",
|
|
246
|
+
"cpu interval 6: 29.02% (ps %cpu sum for reference 64.3%) pids=9",
|
|
247
|
+
"idle rss total=1414MB chrome=1290.1MB node=123.9MB procs=10",
|
|
248
|
+
"tab_switch total 3416ms (open2 3392, back+read 24)",
|
|
249
|
+
"control second tab to example.com 65ms",
|
|
250
|
+
"kill tree pids 11, chrome roots 1, survivors after kill 1",
|
|
251
|
+
"restart_ms 5133ms (kill -> navigate return), cold run2 2755ms",
|
|
252
|
+
"cold_start run3 2464ms"
|
|
253
|
+
],
|
|
254
|
+
"error": null
|
|
255
|
+
},
|
|
256
|
+
"chrome_devtools_mcp": {
|
|
257
|
+
"package": "chrome-devtools-mcp",
|
|
258
|
+
"package_version": "1.12.0",
|
|
259
|
+
"launch_command": "npx -y chrome-devtools-mcp@latest --headless --chromeArg=--use-mock-keychain",
|
|
260
|
+
"mcp_server_info": {
|
|
261
|
+
"name": "chrome_devtools",
|
|
262
|
+
"title": "Chrome DevTools MCP server",
|
|
263
|
+
"version": "1.12.0"
|
|
264
|
+
},
|
|
265
|
+
"cold_start_ms": {
|
|
266
|
+
"runs_ms": [
|
|
267
|
+
1596,
|
|
268
|
+
2185,
|
|
269
|
+
2337
|
|
270
|
+
],
|
|
271
|
+
"min": 1596,
|
|
272
|
+
"median": 2185,
|
|
273
|
+
"max": 2337,
|
|
274
|
+
"definition": "wall clock from spawn(npx) to the navigate tool call returning the page URL, npx package cache warm (already downloaded). For chrome-devtools-mcp this includes the mandatory list_pages call that the tool schema requires to obtain a pageId; that extra round trip is inherent to its API."
|
|
275
|
+
},
|
|
276
|
+
"first_npx_ms": 17891,
|
|
277
|
+
"first_npx_ms_method": "wall clock of the very first `npx -y <pkg>@latest --help` on a cold npm cache (download + install + start); later runs reuse the npx cache",
|
|
278
|
+
"idle_rss_mb": {
|
|
279
|
+
"total_mb": 1279.8,
|
|
280
|
+
"chrome_tree_mb": 1108.8,
|
|
281
|
+
"node_npx_mb": 171,
|
|
282
|
+
"chrome_process_count": 7,
|
|
283
|
+
"total_process_count": 10,
|
|
284
|
+
"chrome_processes": [
|
|
285
|
+
{
|
|
286
|
+
"pid": 29867,
|
|
287
|
+
"rss_mb": 512.5
|
|
288
|
+
},
|
|
289
|
+
{
|
|
290
|
+
"pid": 30891,
|
|
291
|
+
"rss_mb": 132.5
|
|
292
|
+
},
|
|
293
|
+
{
|
|
294
|
+
"pid": 29877,
|
|
295
|
+
"rss_mb": 128
|
|
296
|
+
},
|
|
297
|
+
{
|
|
298
|
+
"pid": 29871,
|
|
299
|
+
"rss_mb": 102
|
|
300
|
+
},
|
|
301
|
+
{
|
|
302
|
+
"pid": 29872,
|
|
303
|
+
"rss_mb": 83.1
|
|
304
|
+
},
|
|
305
|
+
{
|
|
306
|
+
"pid": 29876,
|
|
307
|
+
"rss_mb": 79.9
|
|
308
|
+
},
|
|
309
|
+
{
|
|
310
|
+
"pid": 29873,
|
|
311
|
+
"rss_mb": 64.2
|
|
312
|
+
}
|
|
313
|
+
],
|
|
314
|
+
"method": "sum of `ps -Ao rss=` (KB) over every descendant of the spawned npx pid, 180 s after navigate returned; chrome_tree_mb restricts that sum to the subtree rooted at the launched Google Chrome binary. macOS RSS counts private + shared mapped pages per process, so summing over Chrome helpers (each mapping the same Chrome Framework) overstates true memory use; the per-process list is included so the double counting is visible. No dedup is applied because the task asks for the RSS sum."
|
|
315
|
+
},
|
|
316
|
+
"restart_ms": 4585,
|
|
317
|
+
"tab_switch_ms": {
|
|
318
|
+
"total_ms": 2918,
|
|
319
|
+
"open_second_page_ms": 2903,
|
|
320
|
+
"switch_back_and_read_ms": 15,
|
|
321
|
+
"second_page_url": "https://en.wikipedia.org/wiki/Main_Page",
|
|
322
|
+
"second_page_loaded": true,
|
|
323
|
+
"read_ok": true,
|
|
324
|
+
"control_open_second_page_same_tool_example_com_ms": 151
|
|
325
|
+
},
|
|
326
|
+
"idle_cpu_pct": {
|
|
327
|
+
"avg_pct_all_windows": 36.29,
|
|
328
|
+
"steady_state_avg_pct_after_60s": 38.87,
|
|
329
|
+
"peak_window_pct": 60.27,
|
|
330
|
+
"windows": [
|
|
331
|
+
{
|
|
332
|
+
"window": "0-30s after navigate",
|
|
333
|
+
"avg_cpu_pct_of_one_core": 4.57,
|
|
334
|
+
"pids": 11,
|
|
335
|
+
"top": [
|
|
336
|
+
{
|
|
337
|
+
"pid": 34503,
|
|
338
|
+
"sec": 1.26
|
|
339
|
+
},
|
|
340
|
+
{
|
|
341
|
+
"pid": 34510,
|
|
342
|
+
"sec": 0.03
|
|
343
|
+
},
|
|
344
|
+
{
|
|
345
|
+
"pid": 34509,
|
|
346
|
+
"sec": 0.02
|
|
347
|
+
}
|
|
348
|
+
]
|
|
349
|
+
},
|
|
350
|
+
{
|
|
351
|
+
"window": "30-60s after navigate",
|
|
352
|
+
"avg_cpu_pct_of_one_core": 60.27,
|
|
353
|
+
"pids": 10,
|
|
354
|
+
"top": [
|
|
355
|
+
{
|
|
356
|
+
"pid": 34503,
|
|
357
|
+
"sec": 18.07
|
|
358
|
+
},
|
|
359
|
+
{
|
|
360
|
+
"pid": 34510,
|
|
361
|
+
"sec": 0.01
|
|
362
|
+
}
|
|
363
|
+
]
|
|
364
|
+
},
|
|
365
|
+
{
|
|
366
|
+
"window": "60-90s after navigate",
|
|
367
|
+
"avg_cpu_pct_of_one_core": 3.77,
|
|
368
|
+
"pids": 9,
|
|
369
|
+
"top": [
|
|
370
|
+
{
|
|
371
|
+
"pid": 34503,
|
|
372
|
+
"sec": 1.1
|
|
373
|
+
},
|
|
374
|
+
{
|
|
375
|
+
"pid": 34509,
|
|
376
|
+
"sec": 0.01
|
|
377
|
+
},
|
|
378
|
+
{
|
|
379
|
+
"pid": 34510,
|
|
380
|
+
"sec": 0.01
|
|
381
|
+
}
|
|
382
|
+
]
|
|
383
|
+
},
|
|
384
|
+
{
|
|
385
|
+
"window": "90-120s after navigate",
|
|
386
|
+
"avg_cpu_pct_of_one_core": 56.87,
|
|
387
|
+
"pids": 9,
|
|
388
|
+
"top": [
|
|
389
|
+
{
|
|
390
|
+
"pid": 34503,
|
|
391
|
+
"sec": 16.93
|
|
392
|
+
},
|
|
393
|
+
{
|
|
394
|
+
"pid": 34466,
|
|
395
|
+
"sec": 0.06
|
|
396
|
+
},
|
|
397
|
+
{
|
|
398
|
+
"pid": 34500,
|
|
399
|
+
"sec": 0.06
|
|
400
|
+
}
|
|
401
|
+
]
|
|
402
|
+
},
|
|
403
|
+
{
|
|
404
|
+
"window": "120-150s after navigate",
|
|
405
|
+
"avg_cpu_pct_of_one_core": 55.97,
|
|
406
|
+
"pids": 9,
|
|
407
|
+
"top": [
|
|
408
|
+
{
|
|
409
|
+
"pid": 34503,
|
|
410
|
+
"sec": 16.78
|
|
411
|
+
},
|
|
412
|
+
{
|
|
413
|
+
"pid": 34512,
|
|
414
|
+
"sec": 0.01
|
|
415
|
+
}
|
|
416
|
+
]
|
|
417
|
+
}
|
|
418
|
+
],
|
|
419
|
+
"cross_check_bench_run_avg_pct": 33.55,
|
|
420
|
+
"cross_check_top_instantaneous_pct": [
|
|
421
|
+
{
|
|
422
|
+
"window": "0-15s",
|
|
423
|
+
"delta_method_pct": 28.7,
|
|
424
|
+
"top_l2_pct": 0
|
|
425
|
+
},
|
|
426
|
+
{
|
|
427
|
+
"window": "15-30s",
|
|
428
|
+
"delta_method_pct": 59.6,
|
|
429
|
+
"top_l2_pct": 49.6
|
|
430
|
+
},
|
|
431
|
+
{
|
|
432
|
+
"window": "30-45s",
|
|
433
|
+
"delta_method_pct": 26.5,
|
|
434
|
+
"top_l2_pct": 45.1
|
|
435
|
+
},
|
|
436
|
+
{
|
|
437
|
+
"window": "45-60s",
|
|
438
|
+
"delta_method_pct": 0.2,
|
|
439
|
+
"top_l2_pct": 0
|
|
440
|
+
},
|
|
441
|
+
{
|
|
442
|
+
"window": "60-75s",
|
|
443
|
+
"delta_method_pct": 64,
|
|
444
|
+
"top_l2_pct": 61.4
|
|
445
|
+
},
|
|
446
|
+
{
|
|
447
|
+
"window": "75-90s",
|
|
448
|
+
"delta_method_pct": 50.6,
|
|
449
|
+
"top_l2_pct": 48.3
|
|
450
|
+
},
|
|
451
|
+
{
|
|
452
|
+
"window": "90-105s",
|
|
453
|
+
"delta_method_pct": 29.7,
|
|
454
|
+
"top_l2_pct": 0
|
|
455
|
+
},
|
|
456
|
+
{
|
|
457
|
+
"window": "105-120s",
|
|
458
|
+
"delta_method_pct": 86.7,
|
|
459
|
+
"top_l2_pct": 53.4
|
|
460
|
+
}
|
|
461
|
+
],
|
|
462
|
+
"method_controls": {
|
|
463
|
+
"sleep_process_cpu_seconds": 0,
|
|
464
|
+
"note": "The same ps-time delta applied to a plain `sleep 8` yields 0.00 CPU s, and top -l 2 (instantaneous per-process %CPU) on the Chrome main process tracks the delta method within sampling noise, so the numbers are CPU actually burned by the tree and not an accounting artifact."
|
|
465
|
+
},
|
|
466
|
+
"error": null,
|
|
467
|
+
"method": "Instantaneous CPU of the benchmark process tree, not the ps %cpu column. Every 30 s the cumulative CPU seconds of all tree pids are read from `ps -Ao pid=,time=`, and the delta divided by the 30 s wall window gives the average fraction of one core used by the whole tree (npx + node + all Chrome processes). Each window starts where the previous ended, from navigate +0 s; the early windows include Chrome post-load work and the ones after 60 s are the settled state. The ps %cpu column is a lifetime average and inflates for short-lived helpers, so it is only kept for reference."
|
|
468
|
+
},
|
|
469
|
+
"auto_release": {
|
|
470
|
+
"has_mechanism": false,
|
|
471
|
+
"parameter": null,
|
|
472
|
+
"default_behavior": "browser starts on the first tool call that needs it and stays alive until the MCP client disconnects the CDP session or the server process exits; no idle timer",
|
|
473
|
+
"evidence": [
|
|
474
|
+
"https://github.com/ChromeDevTools/chrome-devtools-mcp/blob/main/README.md (README.md:101 in chrome-devtools-mcp@1.12.0): \"The MCP server will start the browser automatically once the MCP client uses a tool that requires a running browser instance.\"",
|
|
475
|
+
"`chrome-devtools-mcp --help` (v1.12.0) lists no idle/close/timeout flag; the only flag mentioning cleanup is --isolated (\"temporary user-data-dir that is automatically cleaned up after the browser is closed\").",
|
|
476
|
+
"build/src/BrowserManager.js:476 in chrome-devtools-mcp@1.12.0 defines #closeBrowser(), reached from the shutdown path (line 184) and from the disconnected handler (#evictIfCurrent, line 442); nothing schedules it on idleness."
|
|
477
|
+
],
|
|
478
|
+
"empirical": {
|
|
479
|
+
"flag_used": null,
|
|
480
|
+
"browser_auto_closed": null,
|
|
481
|
+
"details": "no flag exists to shorten a timeout, so this side was established from --help and source only, not by waiting an hour."
|
|
482
|
+
}
|
|
483
|
+
},
|
|
484
|
+
"notes": [
|
|
485
|
+
"cold_start run1 1596ms",
|
|
486
|
+
"cpu interval 1: 63.49% (ps %cpu sum for reference 62.8%) pids=10",
|
|
487
|
+
"cpu interval 2: 59.24% (ps %cpu sum for reference 63.4%) pids=10",
|
|
488
|
+
"cpu interval 3: 56.53% (ps %cpu sum for reference 50%) pids=10",
|
|
489
|
+
"cpu interval 4: 21.45% (ps %cpu sum for reference 0.4%) pids=10",
|
|
490
|
+
"cpu interval 5: 0.3% (ps %cpu sum for reference 0%) pids=9",
|
|
491
|
+
"cpu interval 6: 0.3% (ps %cpu sum for reference 0%) pids=9",
|
|
492
|
+
"idle rss total=1279.8MB chrome=1108.8MB node=171MB procs=10",
|
|
493
|
+
"tab_switch total 2918ms (open2 2903, back+read 15)",
|
|
494
|
+
"control second tab to example.com 151ms",
|
|
495
|
+
"kill tree pids 12, chrome roots 1, survivors after kill 1",
|
|
496
|
+
"restart_ms 4585ms (kill -> navigate return), cold run2 2185ms",
|
|
497
|
+
"cold_start run3 2337ms"
|
|
498
|
+
],
|
|
499
|
+
"error": null
|
|
500
|
+
}
|
|
501
|
+
},
|
|
502
|
+
"method": {
|
|
503
|
+
"overview": "macOS arm64, one competitor at a time, no other Chrome running. Each competitor was driven over MCP stdio (newline-delimited JSON-RPC: initialize with protocolVersion 2024-11-05, notifications/initialized, then tools/call) by a ~200-line Node client with no dependencies (/tmp/compbench/mcp_client.mjs). Both were launched exactly the way a user would launch them with npx, headless, plus --use-mock-keychain so macOS does not open a keychain dialog. The navigated page was always https://example.com.",
|
|
504
|
+
"steps": {
|
|
505
|
+
"first_npx_ms": "Before any measurement, `time npx -y <pkg>@latest --help` was run once for each package against a cold npm cache; that number is first_npx_ms (download + install + start). All later runs reused the npx cache, so cold_start_ms excludes the download.",
|
|
506
|
+
"package_version": "The version npx resolves is what `npm view <pkg> version` reports at measurement time: @playwright/mcp 0.0.84, chrome-devtools-mcp 1.12.0. The MCP serverInfo is recorded alongside it.",
|
|
507
|
+
"cold_start_ms": "Timer starts immediately before spawn(npx) and stops when the navigate tool call returns. Three repetitions per competitor, each from a freshly spawned server; min/median/max over the three. For chrome-devtools-mcp the tool schema requires a pageId, so the flow includes one list_pages round trip before navigate_page.",
|
|
508
|
+
"idle_rss_mb": "After navigate returned the process tree was left alone for 180 s, then every descendant of the spawned npx pid was read with `ps -Ao pid=,ppid=,rss=` and RSS summed (total_mb). chrome_tree_mb is the same sum restricted to the subtree rooted at the launched Google Chrome binary; node_npx_mb is the remainder. macOS RSS counts shared mapped pages in every process, so a Chrome helper set that maps the same framework inflates the sum; the per-process list is kept in the report so that double counting stays visible, and no dedup is applied because the requested metric is the RSS sum.",
|
|
509
|
+
"idle_cpu_pct": "Cumulative CPU seconds (`ps -Ao time=`) of every tree pid were sampled every 30 s and differenced, giving the fraction of one core burned by npx+node+Chrome in each window, from the moment navigate returned. The method was validated twice: a plain `sleep 8` shows 0.00 CPU s under the same delta, and `top -l 2` (instantaneous %CPU) on the Chrome main process matches the delta values window by window. The ps %cpu column was not used because it is a lifetime average. The measured CPU is not near zero even with no tool call issued: the Chrome main process alone swings between ~0 and ~87 % of one core, which is Chrome background work, not benchmark overhead.",
|
|
510
|
+
"tab_switch_ms": "With example.com already open, a second page was opened at https://en.wikipedia.org/wiki/Main_Page (Playwright MCP: browser_tabs new + browser_navigate; chrome-devtools-mcp: new_page), then the browser switched back to the first page and read it (browser_tabs select index 0 + browser_snapshot / list_pages + select_page + take_snapshot). total_ms covers the whole cycle. Because wikipedia load time dominates, the same \"open a second page\" call was repeated against example.com as a control, which isolates tool overhead from page load.",
|
|
511
|
+
"restart_ms": "The whole spawned tree was SIGKILLed deepest-first, the death was verified with a fresh ps snapshot (any survivor re-killed), then a new npx process was spawned, initialized and navigated to example.com; restart_ms is kill-start to navigate-return.",
|
|
512
|
+
"auto_release": "Documented behaviour was read from the shipped README and source of the exact installed versions, with line numbers. For Playwright MCP the idle timeout was additionally verified empirically: server started with --idle-timeout=6000, navigate, then 14 s without any tool call, then a fresh process-tree snapshot to see whether Chrome was gone.",
|
|
513
|
+
"process_hygiene": "Every kill was scoped to the descendants of the npx pid this benchmark spawned (tree walk from ps ppid, deepest first) plus the Chrome roots recorded in that tree; no name-based kill was used against system Chrome, and after each kill a fresh snapshot verified that nothing was left."
|
|
514
|
+
},
|
|
515
|
+
"failure_points": [
|
|
516
|
+
"chrome-devtools-mcp navigate_page requires a pageId, and the id is 1-based: pageId 0 returns \"Error: No page found\". Its list_pages result is human-formatted Markdown (\"1: Example Domain (https://example.com/) [selected]\"), not JSON, so the id has to be parsed from the text. Two early probe runs failed on these two points before the driver was fixed.",
|
|
517
|
+
"An early smoke test cleaned up by SIGKILLing every process whose command line matched \"Contents/MacOS/Google Chrome\"; that killed the user's own Chrome main process (pid 97712) instead of only the benchmark tree. It was replaced by the tree-scoped kill now used everywhere. Consequence for this report: no user Chrome was running for any of the measurements below, which removes browser noise from CPU and memory, and the machine was otherwise idle.",
|
|
518
|
+
"A first version of the idle CPU metric summed the ps %cpu column; with freshly forked Chrome helpers that column reports a lifetime average and produced implausible values (up to 98.7 % in an early run). It was replaced by the CPU-seconds-delta method.",
|
|
519
|
+
"wikipedia.org load time varied between runs (3.4 s and 2.9 s in the reported run, 10 s in an earlier run on the same machine and network), which is why tab_switch_ms is reported together with its example.com control and not as tool overhead.",
|
|
520
|
+
"No measurement required a system-level change; nothing outside /tmp/compbench was written and no project repository was touched."
|
|
521
|
+
]
|
|
522
|
+
}
|
|
523
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-inner-browser-use",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.47.0",
|
|
4
4
|
"description": "An agent-owned Chromium sidecar mirrored into the DeepSeek Harness right column: the agent drives it, the person watches and can take the mouse, and logins survive restarts.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "index.js",
|
|
@@ -35,6 +35,7 @@
|
|
|
35
35
|
"scripts/check-sites.mjs",
|
|
36
36
|
"scripts/soak.mjs",
|
|
37
37
|
"scripts/check-rest.mjs",
|
|
38
|
+
"scripts/bench-resource.mjs",
|
|
38
39
|
"runtime/",
|
|
39
40
|
"docs/",
|
|
40
41
|
"README.md",
|
|
@@ -75,7 +76,8 @@
|
|
|
75
76
|
"look": "node scripts/check-look.mjs",
|
|
76
77
|
"sites": "node scripts/check-sites.mjs",
|
|
77
78
|
"soak": "node scripts/soak.mjs",
|
|
78
|
-
"rest": "node scripts/check-rest.mjs"
|
|
79
|
+
"rest": "node scripts/check-rest.mjs",
|
|
80
|
+
"bench": "node scripts/bench-resource.mjs"
|
|
79
81
|
},
|
|
80
82
|
"license": "MIT",
|
|
81
83
|
"author": {
|
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* What this browser costs, in the same terms the neighbours are measured in.
|
|
4
|
+
*
|
|
5
|
+
* node scripts/bench-resource.mjs [--json out.json]
|
|
6
|
+
*
|
|
7
|
+
* Cold start, action round trip, idle memory and CPU with a page open, the
|
|
8
|
+
* reversible freeze, the rest, and the wake. One process tree, one page, the
|
|
9
|
+
* same protocol `docs/RESOURCES.md` uses for Playwright MCP and Chrome DevTools
|
|
10
|
+
* MCP — so the numbers can sit in one table.
|
|
11
|
+
*/
|
|
12
|
+
import { createServer } from 'node:http'
|
|
13
|
+
import { execFileSync } from 'node:child_process'
|
|
14
|
+
import { mkdtempSync, readFileSync, writeFileSync } from 'node:fs'
|
|
15
|
+
import { tmpdir } from 'node:os'
|
|
16
|
+
import { join } from 'node:path'
|
|
17
|
+
|
|
18
|
+
process.env.DSH_INNER_BROWSER_STATE_DIR = mkdtempSync(join(tmpdir(), 'dsh-bench-'))
|
|
19
|
+
const sink = (() => {
|
|
20
|
+
const at = process.argv.indexOf('--json')
|
|
21
|
+
return at === -1 ? null : process.argv[at + 1]
|
|
22
|
+
})()
|
|
23
|
+
|
|
24
|
+
const server = createServer((request, response) => {
|
|
25
|
+
response.writeHead(200, { 'content-type': 'text/html; charset=utf-8' })
|
|
26
|
+
response.end('<!doctype html><meta charset="utf-8"><title>bench</title><h1>bench</h1>' +
|
|
27
|
+
`<p>${'the page has to cost what a page costs. '.repeat(40)}</p>` +
|
|
28
|
+
'<ul>' + Array.from({ length: 60 }, (_, i) => `<li><a href="/row-${i}">row ${i}</a></li>`).join('') + '</ul>')
|
|
29
|
+
})
|
|
30
|
+
await new Promise((done) => server.listen(0, '127.0.0.1', done))
|
|
31
|
+
const url = `http://127.0.0.1:${server.address().port}/page`
|
|
32
|
+
|
|
33
|
+
const { Sidecar } = await import('../sidecar.js')
|
|
34
|
+
const { sidecarTools, sessionScope, workScope } = await import('../tools.js')
|
|
35
|
+
|
|
36
|
+
const wait = (ms) => new Promise((settle) => setTimeout(settle, ms))
|
|
37
|
+
const sidecar = new Sidecar({ profile: 'bench-resource', headless: true, restFloor: 1200, restCeiling: 1600, budgetMb: 4000 })
|
|
38
|
+
const tools = sidecarTools(sidecar, { byChat: new Map() }, {})
|
|
39
|
+
const tool = (name) => tools.find((one) => one.name === name)
|
|
40
|
+
const cwd = mkdtempSync(join(tmpdir(), 'dsh-bench-cwd-'))
|
|
41
|
+
const chat = 'session-bench-resource'
|
|
42
|
+
const run = (name, args) => sessionScope.run(chat, () => workScope.run(cwd, () => tool(name).execute(args, { agent: { id: chat } })))
|
|
43
|
+
const text = (said) => String(said?.text ?? '')
|
|
44
|
+
|
|
45
|
+
/* ps once per sample: pid, parent, cpu, rss. macOS sums shared pages per
|
|
46
|
+
process, so the total overstates real memory; the neighbours are measured the
|
|
47
|
+
same way, which is the only way the numbers compare. */
|
|
48
|
+
function tree(root) {
|
|
49
|
+
const rows = execFileSync('ps', ['-Ao', 'pid=,ppid=,%cpu=,rss='], { encoding: 'utf8' })
|
|
50
|
+
.trim().split('\n').map((line) => line.trim().split(/\s+/)).map(([pid, ppid, cpu, rss]) => ({ pid: Number(pid), ppid: Number(ppid), cpu: Number(cpu), rss: Number(rss) }))
|
|
51
|
+
const byParent = new Map()
|
|
52
|
+
for (const row of rows) {
|
|
53
|
+
if (!byParent.has(row.ppid)) byParent.set(row.ppid, [])
|
|
54
|
+
byParent.get(row.ppid).push(row)
|
|
55
|
+
}
|
|
56
|
+
const out = []
|
|
57
|
+
const walk = (pid) => {
|
|
58
|
+
for (const child of byParent.get(pid) ?? []) {
|
|
59
|
+
out.push(child)
|
|
60
|
+
walk(child.pid)
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
walk(root)
|
|
64
|
+
return out
|
|
65
|
+
}
|
|
66
|
+
function weigh() {
|
|
67
|
+
const root = sidecar.child?.pid
|
|
68
|
+
if (!root) return { mb: 0, cpu: 0, procs: 0 }
|
|
69
|
+
const rows = tree(root)
|
|
70
|
+
return {
|
|
71
|
+
mb: Math.round(rows.reduce((sum, row) => sum + row.rss, 0) / 1024),
|
|
72
|
+
cpu: Number(rows.reduce((sum, row) => sum + row.cpu, 0).toFixed(1)),
|
|
73
|
+
procs: rows.length + 1,
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
async function cpuOver(ms, every = 5000) {
|
|
77
|
+
const samples = []
|
|
78
|
+
const until = Date.now() + ms
|
|
79
|
+
while (Date.now() < until) {
|
|
80
|
+
samples.push(weigh().cpu)
|
|
81
|
+
await wait(Math.min(every, Math.max(0, until - Date.now())))
|
|
82
|
+
}
|
|
83
|
+
return {
|
|
84
|
+
samples_pct: samples,
|
|
85
|
+
avg_pct: Number((samples.reduce((a, b) => a + b, 0) / Math.max(1, samples.length)).toFixed(2)),
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
const report = { generated_at: new Date().toISOString(), host: { arch: process.arch, platform: process.platform }, version: JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8')).version }
|
|
90
|
+
let failed = 0
|
|
91
|
+
const check = (ok, what) => {
|
|
92
|
+
console.log(`${ok ? 'ok ' : 'FAIL'} ${what}`)
|
|
93
|
+
if (!ok) failed += 1
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
try {
|
|
97
|
+
/* Keep the rest out of the way while the page is measured. */
|
|
98
|
+
const coldAt = Date.now()
|
|
99
|
+
await run('browser_open', { url })
|
|
100
|
+
const first = text(await run('browser_view', {}))
|
|
101
|
+
report.cold_start_ms = Date.now() - coldAt
|
|
102
|
+
check(first.includes('bench'), `cold start ${String(report.cold_start_ms)} ms`)
|
|
103
|
+
check(report.cold_start_ms < 6000, 'cold start under six seconds')
|
|
104
|
+
|
|
105
|
+
const readAt = Date.now()
|
|
106
|
+
const read = text(await run('browser_read', {}))
|
|
107
|
+
report.read_ms = Date.now() - readAt
|
|
108
|
+
report.read_chars = read.length
|
|
109
|
+
check(report.read_ms < 4000 && read.length > 200, `a read is ${String(report.read_ms)} ms and ${String(report.read_chars)} chars`)
|
|
110
|
+
|
|
111
|
+
const evalAt = Date.now()
|
|
112
|
+
for (let i = 0; i < 5; i += 1) await run('browser_eval', { code: 'document.title' })
|
|
113
|
+
report.action_rtt_ms = Math.round((Date.now() - evalAt) / 5)
|
|
114
|
+
check(report.action_rtt_ms < 500, `an action round trip is ${String(report.action_rtt_ms)} ms`)
|
|
115
|
+
|
|
116
|
+
const up = weigh()
|
|
117
|
+
report.open_rss_mb = up.mb
|
|
118
|
+
report.open_procs = up.procs
|
|
119
|
+
check(up.mb > 0, `a page open costs ${String(up.mb)} MB in ${String(up.procs)} processes`)
|
|
120
|
+
|
|
121
|
+
/* The first seconds after a load are the page finishing; the number that
|
|
122
|
+
matters is what it costs once it has finished. */
|
|
123
|
+
sidecar.touch(600_000)
|
|
124
|
+
await wait(5000)
|
|
125
|
+
report.idle_cpu = await cpuOver(30_000)
|
|
126
|
+
sidecar.touch(600_000)
|
|
127
|
+
await wait(2000)
|
|
128
|
+
const idle = weigh()
|
|
129
|
+
report.idle_rss_mb = idle.mb
|
|
130
|
+
check(true, `idle 30 s: ${String(idle.mb)} MB, ${String(report.idle_cpu.avg_pct)} % cpu`)
|
|
131
|
+
|
|
132
|
+
/* The reversible park: the page keeps its DOM, the process keeps its share. */
|
|
133
|
+
const frozenAt = Date.now()
|
|
134
|
+
await sidecar.pageFor(chat)?.freeze()
|
|
135
|
+
await wait(3000)
|
|
136
|
+
const frozen = weigh()
|
|
137
|
+
report.freeze_ms = Date.now() - frozenAt
|
|
138
|
+
report.frozen_rss_mb = frozen.mb
|
|
139
|
+
const frozenCpu = await cpuOver(6000)
|
|
140
|
+
report.frozen_cpu_pct = frozenCpu.avg_pct
|
|
141
|
+
check(report.frozen_rss_mb <= idle.mb + 8, `frozen costs ${String(report.frozen_rss_mb)} MB, was ${String(idle.mb)} MB`)
|
|
142
|
+
|
|
143
|
+
const thawAt = Date.now()
|
|
144
|
+
await sidecar.pageFor(chat)?.thaw()
|
|
145
|
+
const quick = text(await run('browser_view', {}))
|
|
146
|
+
report.thaw_ms = Date.now() - thawAt
|
|
147
|
+
check(quick.includes('bench') || quick.includes('nothing changed'), `thaw took ${String(report.thaw_ms)} ms`)
|
|
148
|
+
|
|
149
|
+
/* The full release and the wake that costs a cold launch. */
|
|
150
|
+
sidecar.touch(600)
|
|
151
|
+
for (let i = 0; i < 60; i += 1) {
|
|
152
|
+
await wait(200)
|
|
153
|
+
if ((await sidecar.state()).alive !== true) break
|
|
154
|
+
}
|
|
155
|
+
const asleep = await sidecar.state()
|
|
156
|
+
report.rested = asleep.alive !== true
|
|
157
|
+
report.asleep_mb = asleep.memory
|
|
158
|
+
check(report.rested && asleep.memory === 0, `rest releases everything (${String(asleep.memory)} bytes)`)
|
|
159
|
+
|
|
160
|
+
const wakeAt = Date.now()
|
|
161
|
+
const back = text(await run('browser_view', {}))
|
|
162
|
+
report.wake_ms = Date.now() - wakeAt
|
|
163
|
+
check(sidecar.pageFor(chat)?.url === url, `wake came back to ${String(sidecar.pageFor(chat)?.url)} in ${String(report.wake_ms)} ms`)
|
|
164
|
+
check(back.length > 0, 'the page still reads after the wake')
|
|
165
|
+
} finally {
|
|
166
|
+
server.close()
|
|
167
|
+
await sidecar.shutdown().catch(() => {})
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
console.log('\n' + JSON.stringify(report, null, 2))
|
|
171
|
+
if (sink !== null) writeFileSync(sink, JSON.stringify(report, null, 2) + '\n')
|
|
172
|
+
console.log(failed === 0 ? '\nresource numbers hold.' : `\n${String(failed)} bench checks failed.`)
|
|
173
|
+
process.exitCode = failed === 0 ? 0 : 1
|
package/sidecar.js
CHANGED
|
@@ -1504,9 +1504,26 @@ export class SidecarPage {
|
|
|
1504
1504
|
* a launch and a sign-in. Chrome's own docs define the two states.
|
|
1505
1505
|
* @returns {Promise<void>} done, even on a page that cannot be frozen.
|
|
1506
1506
|
*/
|
|
1507
|
+
/**
|
|
1508
|
+
* What a parked page gives back: the heap it grew while it was being read,
|
|
1509
|
+
* and the decoded images and back/forward caches the renderer keeps for a
|
|
1510
|
+
* page a person may look at again. Frozen pages cannot run the collector, so
|
|
1511
|
+
* this happens before the freeze — the DOM, the scroll position and the
|
|
1512
|
+
* cookies are untouched, which is the whole point of parking instead of
|
|
1513
|
+
* closing.
|
|
1514
|
+
* @returns {Promise<void>} done, even on a renderer that refuses.
|
|
1515
|
+
*/
|
|
1516
|
+
async shed() {
|
|
1517
|
+
await this.cdp.send('Runtime.evaluate', {
|
|
1518
|
+
expression: 'globalThis.gc?.()', silent: true, returnByValue: true,
|
|
1519
|
+
}).catch(() => {})
|
|
1520
|
+
await this.cdp.send('Memory.simulatePressureNotification', { level: 'critical' }).catch(() => {})
|
|
1521
|
+
}
|
|
1522
|
+
|
|
1507
1523
|
async freeze() {
|
|
1508
1524
|
if (this.frozen === true) return
|
|
1509
1525
|
try {
|
|
1526
|
+
await this.shed()
|
|
1510
1527
|
await this.cdp.send('Page.setWebLifecycleState', { state: 'frozen' })
|
|
1511
1528
|
this.frozen = true
|
|
1512
1529
|
this.casting = false
|
|
@@ -2381,6 +2398,10 @@ export class Sidecar {
|
|
|
2381
2398
|
this.expired?.delete(id)
|
|
2382
2399
|
const had = this.pageFor(id)
|
|
2383
2400
|
if (had !== null) {
|
|
2401
|
+
/* A parked page is still the chat's page: waking it costs a protocol call,
|
|
2402
|
+
and a tool that read without waking it would see a page that never runs
|
|
2403
|
+
its scripts again. */
|
|
2404
|
+
await had.thaw?.()
|
|
2384
2405
|
if (url !== undefined && url !== '') await had.navigate(url)
|
|
2385
2406
|
return had
|
|
2386
2407
|
}
|
|
@@ -2389,6 +2410,7 @@ export class Sidecar {
|
|
|
2389
2410
|
const underway = this.opening?.get(id)
|
|
2390
2411
|
if (underway !== undefined) {
|
|
2391
2412
|
const page = await underway
|
|
2413
|
+
await page.thaw?.()
|
|
2392
2414
|
const want = String(url ?? '')
|
|
2393
2415
|
if (want !== '' && String(page.url ?? '') !== want) await page.navigate(want)
|
|
2394
2416
|
return page
|
|
@@ -2541,6 +2563,10 @@ export class Sidecar {
|
|
|
2541
2563
|
this.expired?.delete(String(id))
|
|
2542
2564
|
const had = this.pageFor(id)
|
|
2543
2565
|
if (had !== null) {
|
|
2566
|
+
/* A parked page is still the chat's page: waking it costs a protocol call,
|
|
2567
|
+
and a tool that read without waking it would see a page that never runs
|
|
2568
|
+
its scripts again. */
|
|
2569
|
+
await had.thaw?.()
|
|
2544
2570
|
if (url !== undefined && url !== '') await had.navigate(url)
|
|
2545
2571
|
return had
|
|
2546
2572
|
}
|
|
@@ -2865,7 +2891,7 @@ export class Sidecar {
|
|
|
2865
2891
|
'--no-default-browser-check',
|
|
2866
2892
|
'--use-mock-keychain',
|
|
2867
2893
|
'--disable-blink-features=AutomationControlled',
|
|
2868
|
-
'--disable-features=Translate,MediaRouter,OptimizationHints',
|
|
2894
|
+
'--disable-features=Translate,MediaRouter,OptimizationHints,InterestFeedContentSuggestions,CalculateNativeWinOcclusion',
|
|
2869
2895
|
'--disable-sync',
|
|
2870
2896
|
/* The driver clicks for a person, and a page that opens a tab on a click
|
|
2871
2897
|
is doing what the page means to do. Blocking it would strand the
|
|
@@ -2882,6 +2908,24 @@ export class Sidecar {
|
|
|
2882
2908
|
'--disable-renderer-backgrounding',
|
|
2883
2909
|
'--disable-ipc-flooding-protection',
|
|
2884
2910
|
'--metrics-recording-only',
|
|
2911
|
+
/* Nothing here talks to Google while the person is not reading: the
|
|
2912
|
+
update, the crash reporter and the network pings all cost a process,
|
|
2913
|
+
a socket and a wake-up on a page that is only being looked at. */
|
|
2914
|
+
'--disable-background-networking',
|
|
2915
|
+
'--disable-component-update',
|
|
2916
|
+
'--disable-domain-reliability',
|
|
2917
|
+
'--disable-client-side-phishing-detection',
|
|
2918
|
+
'--disable-breakpad',
|
|
2919
|
+
'--disable-crash-reporter',
|
|
2920
|
+
'--no-pings',
|
|
2921
|
+
'--disable-extensions',
|
|
2922
|
+
'--disable-default-apps',
|
|
2923
|
+
/* A page nobody looks at can hand its heap back when it is parked; the
|
|
2924
|
+
renderer needs the flag before it will collect on request. */
|
|
2925
|
+
'--js-flags=--expose-gc',
|
|
2926
|
+
/* Six tabs never need six renderers: the cap keeps the resident cost of a
|
|
2927
|
+
browsing session flat, while four still keeps unrelated sites apart. */
|
|
2928
|
+
'--renderer-process-limit=4',
|
|
2885
2929
|
'--no-service-autorun',
|
|
2886
2930
|
'--password-store=basic',
|
|
2887
2931
|
'--lang=zh-CN',
|
|
@@ -2895,6 +2939,10 @@ export class Sidecar {
|
|
|
2895
2939
|
'--window-position=-3000,-3000',
|
|
2896
2940
|
'about:blank',
|
|
2897
2941
|
]
|
|
2942
|
+
/* An escape hatch for measuring one flag against another without editing
|
|
2943
|
+
* the source: space-separated extra switches, never a replacement. */
|
|
2944
|
+
const extra = String(process.env.DSH_INNER_BROWSER_CHROME_FLAGS ?? '').trim()
|
|
2945
|
+
if (extra !== '') args.push(...extra.split(/\s+/))
|
|
2898
2946
|
if (flash !== null) {
|
|
2899
2947
|
/* Electron takes these switches through its own command line, not the
|
|
2900
2948
|
argv it hands the app, so the runtime app reads them from here. */
|