dsh-inner-browser-use 0.36.0 → 0.38.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/README.zh.md +1 -1
- package/docs/ARCHITECTURE.md +5 -0
- package/docs/ARTIFACTS.md +14 -0
- package/docs/BENCH.md +32 -0
- package/index.js +1 -1
- package/media.js +70 -0
- package/package.json +1 -1
- package/sidecar.js +14 -0
- package/tools.js +108 -21
package/README.md
CHANGED
|
@@ -20,7 +20,7 @@ live picture of that browser, and your own cursor keeps working on the picture.
|
|
|
20
20
|
| `browser_key` | Presses one key: Enter, Tab, Escape, Backspace, Delete, Insert, arrows, Home, End, PageUp, PageDown, F1-F12, the numeric keypad, letters, digits, punctuation and every shifted symbol, plus ContextMenu, CapsLock, PrintScreen and media keys. `Ctrl`/`Alt`/`Shift`/`Meta`/`Mod` modifiers, chords with `+`, and `Ctrl+C/X/V/A/Z/Y` run the real editing commands. |
|
|
21
21
|
| `browser_scroll` | Scrolls the wheel at the centre of the page, or at `x`/`y`. `dy: 600` is about a screen down. |
|
|
22
22
|
| `browser_shot` | Saves a full-size PNG into `~/.dsh-inner-browser-use/screenshots` and returns the path. |
|
|
23
|
-
| `browser_media` | Lists the video and sound the page already fetched, with sizes
|
|
23
|
+
| `browser_media` | Lists the video and sound the page already fetched, with sizes, the one its own player shows first and marked `main`; `save: <row>`, `save: main` or part of the address keeps one. Ads and beacons stay out unless `all: true`; `watch: false` stops remembering the page. A stream is followed to its pieces and joined, so nothing has to be installed. |
|
|
24
24
|
| `browser_state` | Reports whether the browser runs, which profile it uses, its address and title, and how many columns are showing it. |
|
|
25
25
|
|
|
26
26
|
`browser_read` on the same page is the words only, in Markdown (measured: 4,520
|
package/README.zh.md
CHANGED
|
@@ -19,7 +19,7 @@ profile,不是产品自带的那个浏览页签:右栏放的是它的画面
|
|
|
19
19
|
| `browser_key` | 按一个键:Enter、Tab、Escape、Backspace、Delete、上下左右、Home、End、PageUp、PageDown。 |
|
|
20
20
|
| `browser_scroll` | 在页面中心滚轮,也可以指定 `x`/`y`。`dy: 600` 约一屏。 |
|
|
21
21
|
| `browser_shot` | 存一张 PNG 到 `~/.dsh/inner-browser-use/screenshots`,返回路径;`region: [x, y, w, h]` 只截一块(游戏、图表、看局部)。 |
|
|
22
|
-
| `browser_media` |
|
|
22
|
+
| `browser_media` | 列出页面已经请求过的视频和声音(带大小),页面自己播放器里那个排第一、标 `main`;`save: <行号>`、`save: main` 或地址片段存下来。广告和统计默认不出现(`all: true` 才看),`watch: false` 关掉这个页面的记录。流会顺着播放列表把片段取回拼好,不用装任何东西。 |
|
|
23
23
|
| `browser_state` | 报浏览器在不在跑、用的哪个 profile、地址、标题,以及有几个右栏在看它。 |
|
|
24
24
|
|
|
25
25
|
点、输入、按键之后的那一行会多一句结果:`· navigated to …`(换页了)、`· content changed`(页内有变化)、`· no visible change`(点了没动)、Flash 页则是 `· screen updated` / `· no frame change`;`read: true` 时不再多说。
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -160,6 +160,11 @@ Calls that touch the browser are marked not concurrency-safe; reading state, con
|
|
|
160
160
|
requests is. `browser_open`, `browser_view`, `browser_wait` and `browser_shot` carry longer
|
|
161
161
|
timeouts than the rest.
|
|
162
162
|
|
|
163
|
+
`browser_wait` does not poll: one call hands the page a predicate and a `MutationObserver`, and the
|
|
164
|
+
picture comes back on the mutation that satisfies it. A page that changes 137 ms after the wait
|
|
165
|
+
starts answers at 138 ms, not at the next 100 ms tick — the same reason `idle` (a mutation-free
|
|
166
|
+
window) costs one round trip on an animation-heavy page instead of one per frame.
|
|
167
|
+
|
|
163
168
|
Sign-in state lives in the profile, and session cookies — the kind a sign-in usually sets,
|
|
164
169
|
which Chrome would otherwise drop on exit — are saved next to it and restored on the next
|
|
165
170
|
launch, so a restart of the app does not sign the person out.
|
package/docs/ARTIFACTS.md
CHANGED
|
@@ -16,4 +16,18 @@
|
|
|
16
16
|
|
|
17
17
|
调用环境里没有项目或会话信息时,落到 `no-project/no-chat/<种类>/`,不猜。
|
|
18
18
|
|
|
19
|
+
## 整个目录
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
~/.dsh-inner-browser-use/
|
|
23
|
+
artifacts/<项目>/<会话>/<种类>/ 给人看的产出,只在这里(见上)
|
|
24
|
+
chrome/ 浏览器自己的数据:登录态、Cookie、缓存
|
|
25
|
+
flash/ Flash 播放器运行时,第一次用 flash:true 才装
|
|
26
|
+
browser.log 最近一次浏览器启动和页面记录
|
|
27
|
+
asks.log 人问过什么、答了什么
|
|
28
|
+
tabs.json / mirror.json / flash.json / watcher.json 上次开着哪些页
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
`chrome/` 是浏览器进程自己在读写,不要手动改;要清登录态,用 `browser_open` 的档位或插件自己的入口,别删目录里的某个文件。上面这些都在一个根目录下,卸载插件前想留产出就拷 `artifacts/`。
|
|
32
|
+
|
|
19
33
|
测试里用 `DSH_INNER_BROWSER_ARTIFACTS` 指向临时目录,不写真实 home。
|
package/docs/BENCH.md
CHANGED
|
@@ -1,3 +1,35 @@
|
|
|
1
|
+
## 等到变化,而不是去问(0.38.0)
|
|
2
|
+
|
|
3
|
+
`browser_wait` 以前在宿主侧每 100 毫秒读一次页面判断条件,命中与否都被凑到下一个整拍上。
|
|
4
|
+
现在一次调用把谓词和 `MutationObserver` 交给页面,满足条件的那次变化直接 resolve,回程只走
|
|
5
|
+
一次。
|
|
6
|
+
|
|
7
|
+
| 同一件事 | 0.37.0(宿主轮询) | 0.38.0(页面观察) |
|
|
8
|
+
| --- | --- | --- |
|
|
9
|
+
| 五次不同时相的等待,平均耗时 | 205 ms | 140 ms |
|
|
10
|
+
| 每次等待的探测来回 | 3 | 1 |
|
|
11
|
+
| `idle` 判定 | 固定间隔重读页面 | 350 ms 无变化即停,一次来回 |
|
|
12
|
+
|
|
13
|
+
`wait-late` 行就是上表:每次实验在等待开始的同一刻给页面装一个定时器,所以旧路径必然要走到
|
|
14
|
+
下一个 100 毫秒整拍,新路径不必。预算 600 ms,实测 140 ms。
|
|
15
|
+
|
|
16
|
+
## 只列该看的那一个(0.37.0,同一台本地页面,加了两个广告请求)
|
|
17
|
+
|
|
18
|
+
页面自己放一个 `<video src="/movie.mp4">`、一个 `<audio>`、fetch 一个 master 播放列表,再 fetch 两条广告(`/ads/preroll.mp4` 9 MB、`/px/beacon.mp4` 2 B)。不筛选时 5 条都在;默认列表剩 3 条,页面播放器那个标 `main` 排第一,`save: main` 存的就是它。
|
|
19
|
+
|
|
20
|
+
| 项 | 0.36.0 | 0.37.0 |
|
|
21
|
+
|---|---|---|
|
|
22
|
+
| 列表行数(同一页面 5 条请求) | 5 | 3,广告单独一行说数量 |
|
|
23
|
+
| 首行 | `1. video 20 B · /movie.mp4` | `1. main video 156 B · /movie.mp4` |
|
|
24
|
+
| 找人要的那个 | 数行号 / 猜地址 | `save: main` |
|
|
25
|
+
| 列表耗时 | 0 ms(纯内存) | 1 ms(多一次页面查询拿播放器与时长) |
|
|
26
|
+
|
|
27
|
+
`media-pick` 1 ms,预算 `media-pick: 150 ms`;`all: true` 回到 5 行。`media-sniff` 仍 0 ms(记录仍是网络日志附带的),`media-save` 4 ms、96 B、`6 parts joined`,`budget checks 17 ok`。
|
|
28
|
+
|
|
29
|
+
## 媒体嗅探与流保存(0.36.0,本地 http 页面,`scripts/bench.mjs`)
|
|
30
|
+
|
|
31
|
+
页面自己放 `<video>`、`<audio>` 并 fetch 一个 master 播放列表:加载完的那一刻网络记录里已经有 3 条,`media-sniff` 0 ms(没有额外请求,嗅探就是已有的网络记录)。保存 master → 1080 → 6 个片段,`media-save` 5 ms、96 B,报 `6 parts joined`;列表首行 59 字符。预算 `media-sniff: 400 ms`、`media-save: 500 ms` 均通过,`budget checks 16 ok`。
|
|
32
|
+
|
|
1
33
|
## Flash 镜像按栏改窗口(0.34.48,Electron Flash 运行时,右栏 747×1196)
|
|
2
34
|
|
|
3
35
|
Flash 路径原来采集一个固定的 1440×900 窗口,竖高右栏里上下留黑边。现在 `setViewport` 直接改窗口大小,采集源与栏同比例。
|
package/index.js
CHANGED
|
@@ -944,7 +944,7 @@ export function apply(ctx) {
|
|
|
944
944
|
'先接着用已有的页:`browser_view`(拿编号控件与可见文字)→ `browser_click` / `browser_type` → 再 `browser_view` 直到出现要的结果;页面要等就重复读,不 sleep。`browser_open` 只用于没有页、要换新地址、或要明确刷新;同一个地址不要反复重开。',
|
|
945
945
|
'页面很长、只想确认某句话/按钮在哪时用 `browser_find`,别整页读。要读长文(文章、文档、issue、论坛帖)用 `browser_read`:它回 Markdown,只花几十分之一的 token,链接可点;要动手点、填表才用 `browser_view` 拿编号。',
|
|
946
946
|
'要保存页面样子用 `browser_shot`,它会给出 PNG 绝对路径;要交给人看的文件,把绝对路径写进回复。',
|
|
947
|
-
'页面放视频或音乐、人想留下它时用 `browser_media
|
|
947
|
+
'页面放视频或音乐、人想留下它时用 `browser_media`:它列的是页面自己已经请求过的媒体(带大小),页面自己播放器里那个标 `main` 排第一,广告和统计请求默认不出现(`all:true` 才看),`save: main` 存第一个;视频是流的话会顺着播放列表把片段取回拼成整段,不用装 yt-dlp 或 ffmpeg。先让页面播一下再列,播放请求才是文件。不想要这个页面的记录就 `watch:false`。',
|
|
948
948
|
'游戏、canvas、Flash:画面自己画的地方,`browser_view` 和 `browser_read` 读不出里面是什么。先 `browser_prepare` 拿画面位置、帧率、按键是否到达和下一步;之后看 `browser_shot` 的图定位置,按坐标点、按住、按组合键,再拍一张看画面变了没有——变了就是这一步生效了。当年用 Flash 做的游戏用 `browser_open` 带 `flash:true` 在栏内跑,不装播放器、不开窗口;Flash 默认关,`flash:true` 开、`flash:false` 回到普通 Chrome,两边都只在本会话右栏。',
|
|
949
949
|
].join('\n'),
|
|
950
950
|
}), 'dsh-inner-browser-use: system prompt section')
|
package/media.js
CHANGED
|
@@ -237,3 +237,73 @@ export function sizeText(bytes) {
|
|
|
237
237
|
if (n >= 1000) return `${Math.round(n / 1000)} kB`
|
|
238
238
|
return `${String(n)} B`
|
|
239
239
|
}
|
|
240
|
+
|
|
241
|
+
/**
|
|
242
|
+
* Ad servers and ad slots, recognised by themselves rather than by size: an ad
|
|
243
|
+
* is often the biggest file on the page, and a real clip can be small.
|
|
244
|
+
*/
|
|
245
|
+
const AD_HOST = /^(?:(?:[\w-]+\.)*(?:doubleclick\.net|googlesyndication\.com|googleadservices\.com|amazon-adsystem\.com|adsrvr\.org|taboola\.com|outbrain\.com|criteo\.(?:net|com)|scorecardresearch\.com|moatads\.com|adnxs\.com|pubmatic\.com|rubiconproject\.com|openx\.net)$)$/i;
|
|
246
|
+
const AD_SLOT = /^(?:ad|ads|adv|adserver|advert(?:isement|ising)?|pre-?roll|mid-?roll|post-?roll|banner|sponsor|promo|tracking|tracker|pixel|beacon|analytics|telemetry)\d*$/i
|
|
247
|
+
|
|
248
|
+
/**
|
|
249
|
+
* Whether an address is an ad or a measurement call.
|
|
250
|
+
* @param {string} url - the address.
|
|
251
|
+
* @returns {boolean} true when it is worth hiding from a person picking a file.
|
|
252
|
+
*/
|
|
253
|
+
export function adLike(url) {
|
|
254
|
+
let parsed
|
|
255
|
+
try {
|
|
256
|
+
parsed = new URL(String(url))
|
|
257
|
+
} catch {
|
|
258
|
+
return false
|
|
259
|
+
}
|
|
260
|
+
if (AD_HOST.test(parsed.hostname)) return true
|
|
261
|
+
const pieces = parsed.pathname.split('/').filter((piece) => piece !== '')
|
|
262
|
+
const named = pieces.slice(0, -1).some((piece) => AD_SLOT.test(piece))
|
|
263
|
+
if (named) return true
|
|
264
|
+
const file = pieces.length === 0 ? '' : pieces[pieces.length - 1]
|
|
265
|
+
return /^(?:ad|ads|adv|preroll|banner|pixel|beacon|track|tracking)[-_.]/i.test(file)
|
|
266
|
+
}
|
|
267
|
+
|
|
268
|
+
/**
|
|
269
|
+
* How much a row looks like the thing a person came for: the player on the page
|
|
270
|
+
* first, then a real file, and ads or beacons last.
|
|
271
|
+
* @param {{url: string, mime?: string, kind?: string, bytes?: number}} row - one media request.
|
|
272
|
+
* @param {Array<{url: string, dur?: number}>} [players] - what the page's own `<video>` / `<audio>` plays.
|
|
273
|
+
* @returns {number} the score; higher is more likely the point of the page.
|
|
274
|
+
*/
|
|
275
|
+
export function coreScore(row, players = []) {
|
|
276
|
+
const url = String(row?.url ?? '')
|
|
277
|
+
const bytes = Number(row?.bytes ?? 0)
|
|
278
|
+
const kind = String(row?.kind ?? '')
|
|
279
|
+
const mime = String(row?.mime ?? '')
|
|
280
|
+
let score = 0
|
|
281
|
+
const mine = players.some((one) => {
|
|
282
|
+
const said = String(one?.url ?? '')
|
|
283
|
+
return said !== '' && (said === url || url.startsWith(said) || said.startsWith(url))
|
|
284
|
+
})
|
|
285
|
+
if (mine) score += 100
|
|
286
|
+
if (adLike(url)) score -= 80
|
|
287
|
+
if (bytes >= 1_000_000) score += 25
|
|
288
|
+
else if (bytes >= 200_000) score += 10
|
|
289
|
+
else if (bytes > 0 && bytes < 20_000 && kind !== 'stream') score -= 20
|
|
290
|
+
if (kind === 'stream') score += 15
|
|
291
|
+
if (/^video\//i.test(mime)) score += 5
|
|
292
|
+
if (/^audio\//i.test(mime)) score += 3
|
|
293
|
+
return score
|
|
294
|
+
}
|
|
295
|
+
|
|
296
|
+
/**
|
|
297
|
+
* The rows in the order a person would want them, with ads gone unless asked.
|
|
298
|
+
* @param {Array<object>} rows - media rows, any order.
|
|
299
|
+
* @param {Array<{url: string, dur?: number}>} [players] - the page's own players.
|
|
300
|
+
* @param {boolean} [all] - keep the ads and the beacons too.
|
|
301
|
+
* @returns {Array<object>} the rows to show.
|
|
302
|
+
*/
|
|
303
|
+
export function coreRows(rows, players = [], all = false) {
|
|
304
|
+
const ranked = rows.map((row) => ({ row, score: coreScore(row, players) }))
|
|
305
|
+
return ranked
|
|
306
|
+
.filter((one) => all || one.score >= 0)
|
|
307
|
+
.sort((left, right) => right.score - left.score || Number(right.row?.seq ?? 0) - Number(left.row?.seq ?? 0))
|
|
308
|
+
.map((one) => one.row)
|
|
309
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-inner-browser-use",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.38.0",
|
|
4
4
|
"description": "An agent-owned Chromium sidecar mirrored into the DeepSeek Harness right column: the agent drives it, the person watches and can take the mouse, and logins survive restarts.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "index.js",
|
package/sidecar.js
CHANGED
|
@@ -579,6 +579,7 @@ export class SidecarPage {
|
|
|
579
579
|
/* The movie a page plays is one of the requests it already made, so the
|
|
580
580
|
file behind it costs no second look: the network log is the sniff. */
|
|
581
581
|
this.media = new Map()
|
|
582
|
+
this.mediaOn = true
|
|
582
583
|
/* A PDF is a document the mirror cannot draw: Chrome leaves the page blank
|
|
583
584
|
and a model looking at it would think the address was empty. What it is
|
|
584
585
|
and where the bytes went is state, not a guess. */
|
|
@@ -763,6 +764,7 @@ export class SidecarPage {
|
|
|
763
764
|
* @returns {void} nothing.
|
|
764
765
|
*/
|
|
765
766
|
noteMedia(params) {
|
|
767
|
+
if (this.mediaOn === false) return
|
|
766
768
|
const response = params?.response ?? {}
|
|
767
769
|
const url = String(response.url ?? '')
|
|
768
770
|
const mime = String(response.mimeType ?? '')
|
|
@@ -783,6 +785,18 @@ export class SidecarPage {
|
|
|
783
785
|
if (this.media.size > 60) this.media.delete(this.media.keys().next().value)
|
|
784
786
|
}
|
|
785
787
|
|
|
788
|
+
/**
|
|
789
|
+
* Stop or resume remembering media on this page; stopping empties the list so
|
|
790
|
+
* nothing is kept from before the switch.
|
|
791
|
+
* @param {boolean} on - false stops, true resumes.
|
|
792
|
+
* @returns {boolean} what it is set to.
|
|
793
|
+
*/
|
|
794
|
+
mediaWatch(on) {
|
|
795
|
+
this.mediaOn = on !== false
|
|
796
|
+
if (!this.mediaOn) this.media = new Map()
|
|
797
|
+
return this.mediaOn
|
|
798
|
+
}
|
|
799
|
+
|
|
786
800
|
/**
|
|
787
801
|
* What on this page is worth saving, newest first.
|
|
788
802
|
* @returns {Array<{url: string, mime: string, kind: string, bytes: number, status: number}>} the media.
|
package/tools.js
CHANGED
|
@@ -15,7 +15,7 @@ import { homedir } from 'node:os'
|
|
|
15
15
|
import { basename, join } from 'node:path'
|
|
16
16
|
import { mkdir, readdir, stat, unlink, writeFile } from 'node:fs/promises'
|
|
17
17
|
import { VIEW, DESKTOP_UA } from './sidecar.js?v=1'
|
|
18
|
-
import { planOf, gather, sizeText } from './media.js?m=
|
|
18
|
+
import { planOf, gather, sizeText, coreRows } from './media.js?m=2'
|
|
19
19
|
import { readPage, renderPage, readArticle, readRelevant, findOnPage, locate, locateFirst, FRAME_BASE, covering, contextFor, identityOf, sameThing, differs, labelSpot, selectSpot, bounds, pictures, clickInDom, writeInDom, REF_GONE } from './read.js?v=1'
|
|
20
20
|
import { brief as askBrief, recordRound, lastRoundNote, lastRound, markRound, MARK_STATUS } from './review.js?v=1'
|
|
21
21
|
|
|
@@ -37,6 +37,24 @@ function extensionOf(url, mime = '') {
|
|
|
37
37
|
return '.bin'
|
|
38
38
|
}
|
|
39
39
|
|
|
40
|
+
/** What the page's own players are showing, in one look: the address, the length, whether it runs. */
|
|
41
|
+
async function playersOf(page) {
|
|
42
|
+
const answer = await page.cdp.send('Runtime.evaluate', {
|
|
43
|
+
expression: `(() => [...document.querySelectorAll('video,audio')].map((one) => ({ url: one.currentSrc || one.src || '', playing: !one.paused, dur: Number.isFinite(one.duration) ? Math.round(one.duration) : 0 })))()`,
|
|
44
|
+
returnByValue: true,
|
|
45
|
+
}).catch(() => null)
|
|
46
|
+
const seen = answer?.result?.value
|
|
47
|
+
return Array.isArray(seen) ? seen.filter((one) => String(one?.url ?? '') !== '') : []
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
/** Seconds as a person writes them: 2:13, 1:02:00. */
|
|
51
|
+
function clockOf(seconds) {
|
|
52
|
+
const whole = Math.max(0, Math.round(Number(seconds) || 0))
|
|
53
|
+
const rest = whole % 3600
|
|
54
|
+
const pad = (value) => String(value).padStart(2, '0')
|
|
55
|
+
return whole >= 3600 ? `${String(Math.floor(whole / 3600))}:${pad(Math.floor(rest / 60))}:${pad(rest % 60)}` : `${String(Math.floor(whole / 60))}:${pad(rest % 60)}`
|
|
56
|
+
}
|
|
57
|
+
|
|
40
58
|
/** Which chat the running call belongs to, so each gets its own tab. */
|
|
41
59
|
export const sessionScope = new AsyncLocalStorage()
|
|
42
60
|
|
|
@@ -1052,13 +1070,64 @@ export async function settle(page, timeout = 5000, since = 0) {
|
|
|
1052
1070
|
* @returns {Promise<boolean>} whether it ever held.
|
|
1053
1071
|
*/
|
|
1054
1072
|
async function until(page, expression, timeout) {
|
|
1055
|
-
|
|
1056
|
-
|
|
1057
|
-
|
|
1058
|
-
|
|
1059
|
-
|
|
1060
|
-
|
|
1061
|
-
|
|
1073
|
+
/* The page watches itself. One round trip that the page resolves on the
|
|
1074
|
+
mutation which matters: no snapshot every 100 ms, no rounding the answer up
|
|
1075
|
+
to the next tick, and nothing to miss between two polls. */
|
|
1076
|
+
const ceiling = Math.max(200, Number(timeout) || 1000)
|
|
1077
|
+
const script = `new Promise((done) => {
|
|
1078
|
+
const deadline = Date.now() + ${String(ceiling)}
|
|
1079
|
+
let watcher = null
|
|
1080
|
+
let timer = null
|
|
1081
|
+
let frame = null
|
|
1082
|
+
const stop = (value) => {
|
|
1083
|
+
if (watcher !== null) watcher.disconnect()
|
|
1084
|
+
watcher = null
|
|
1085
|
+
if (timer !== null) clearTimeout(timer)
|
|
1086
|
+
timer = null
|
|
1087
|
+
if (frame !== null) cancelAnimationFrame(frame)
|
|
1088
|
+
frame = null
|
|
1089
|
+
done(value)
|
|
1090
|
+
}
|
|
1091
|
+
const test = () => { try { return Boolean(${expression}) } catch (error) { return false } }
|
|
1092
|
+
if (test()) return stop(true)
|
|
1093
|
+
watcher = new MutationObserver(() => { if (test()) stop(true) })
|
|
1094
|
+
if (document.documentElement !== null) watcher.observe(document.documentElement, { childList: true, subtree: true, attributes: true, characterData: true })
|
|
1095
|
+
const tick = () => {
|
|
1096
|
+
if (test()) return stop(true)
|
|
1097
|
+
if (Date.now() >= deadline) return stop(false)
|
|
1098
|
+
frame = requestAnimationFrame(tick)
|
|
1099
|
+
}
|
|
1100
|
+
frame = requestAnimationFrame(tick)
|
|
1101
|
+
timer = setTimeout(() => stop(false), deadline - Date.now())
|
|
1102
|
+
})`
|
|
1103
|
+
const answer = await evaluate(page, script, undefined, ceiling + 3000)
|
|
1104
|
+
return answer === true
|
|
1105
|
+
}
|
|
1106
|
+
|
|
1107
|
+
/**
|
|
1108
|
+
* How long the page has been still: a mutation resets the clock, and the call
|
|
1109
|
+
* comes back the moment nothing has moved for `quiet` ms. Reads no node, so an
|
|
1110
|
+
* animation-heavy page costs the same as a blank one.
|
|
1111
|
+
*/
|
|
1112
|
+
async function quietInPage(page, timeout = 5000, quiet = 350) {
|
|
1113
|
+
const ceiling = Math.max(300, Number(timeout) || 5000)
|
|
1114
|
+
const script = `new Promise((done) => {
|
|
1115
|
+
const ceiling = Date.now() + ${String(ceiling)}
|
|
1116
|
+
const settleMs = ${String(Math.max(50, Number(quiet) || 350))}
|
|
1117
|
+
let changedAt = Date.now()
|
|
1118
|
+
let frame = null
|
|
1119
|
+
const watcher = new MutationObserver(() => { changedAt = Date.now() })
|
|
1120
|
+
if (document.documentElement !== null) watcher.observe(document.documentElement, { childList: true, subtree: true, attributes: true, characterData: true })
|
|
1121
|
+
const tick = () => {
|
|
1122
|
+
const now = Date.now()
|
|
1123
|
+
if (now - changedAt >= settleMs) { watcher.disconnect(); return done(true) }
|
|
1124
|
+
if (now >= ceiling) { watcher.disconnect(); return done(false) }
|
|
1125
|
+
frame = requestAnimationFrame(tick)
|
|
1126
|
+
}
|
|
1127
|
+
frame = requestAnimationFrame(tick)
|
|
1128
|
+
})`
|
|
1129
|
+
const answer = await evaluate(page, script, undefined, ceiling + 3000)
|
|
1130
|
+
return answer === true
|
|
1062
1131
|
}
|
|
1063
1132
|
|
|
1064
1133
|
/**
|
|
@@ -3132,8 +3201,9 @@ export function sidecarTools(sidecar, refs, host = {}) { // ui-copy-ignore
|
|
|
3132
3201
|
const at = await until(page, `location.href.includes(${JSON.stringify(String(args.url))})`, timeout)
|
|
3133
3202
|
return at ? `at ${String(await evaluate(page, 'location.href'))}` : `address never contained ${String(args.url)}`
|
|
3134
3203
|
}
|
|
3135
|
-
await
|
|
3136
|
-
|
|
3204
|
+
const quiet = await quietInPage(page, timeout)
|
|
3205
|
+
if (typeof page.adoptOpened === 'function') await page.adoptOpened().catch(() => '')
|
|
3206
|
+
return quiet ? 'idle' : 'still moving'
|
|
3137
3207
|
}),
|
|
3138
3208
|
},
|
|
3139
3209
|
{
|
|
@@ -3334,12 +3404,14 @@ export function sidecarTools(sidecar, refs, host = {}) { // ui-copy-ignore
|
|
|
3334
3404
|
|
|
3335
3405
|
{
|
|
3336
3406
|
name: 'browser_media', // ui-copy-ignore
|
|
3337
|
-
description: 'What this page played, and how to keep it: lists the video and sound requests the page already made, with
|
|
3407
|
+
description: 'What this page played, and how to keep it: lists the video and sound requests the page already made, the one its own player shows first, with sizes; save one by row number, by part of its address, or save: main. Ads and measurement beacons stay out of the list unless all: true. A stream playlist is followed to its pieces and joined, so nothing has to be installed. watch: false stops remembering this page.',
|
|
3338
3408
|
parameters: {
|
|
3339
3409
|
type: 'object',
|
|
3340
3410
|
properties: {
|
|
3341
3411
|
find: { type: 'string', description: 'Only rows whose address contains this.' },
|
|
3342
|
-
save: { type: ['integer', 'string'], description: 'Row number from the list,
|
|
3412
|
+
save: { type: ['integer', 'string'], description: 'Row number from the list, part of its address, or main for the first row.' },
|
|
3413
|
+
all: { type: 'boolean', description: 'Include ads and beacons too (default false).' },
|
|
3414
|
+
watch: { type: 'boolean', description: 'false stops remembering what this page plays and empties the list; true resumes.' },
|
|
3343
3415
|
height: { type: 'number', description: 'Tallest version to take when one address offers several (default 1080).' },
|
|
3344
3416
|
name: { type: 'string', description: 'File name without extension.' },
|
|
3345
3417
|
},
|
|
@@ -3350,24 +3422,39 @@ export function sidecarTools(sidecar, refs, host = {}) { // ui-copy-ignore
|
|
|
3350
3422
|
isConcurrencySafe: () => false,
|
|
3351
3423
|
/** @returns {Promise<{text: string}>} the list, or where the file went. */
|
|
3352
3424
|
execute: async (args) => withRawPage(async (page) => {
|
|
3353
|
-
|
|
3425
|
+
if (typeof args?.watch === 'boolean') {
|
|
3426
|
+
const on = typeof page.mediaWatch === 'function' ? page.mediaWatch(args.watch) : false
|
|
3427
|
+
if (!on) return 'Off: this page keeps nothing from now on. watch: true puts it back.'
|
|
3428
|
+
if (args.save === undefined || args.save === null || args.save === '') return 'On: ask again once it has played something.'
|
|
3429
|
+
}
|
|
3430
|
+
const known = typeof page.mediaList === 'function' ? page.mediaList() : []
|
|
3431
|
+
const players = known.length === 0 ? [] : await playersOf(page)
|
|
3432
|
+
const ranked = coreRows(known, players, args?.all === true)
|
|
3354
3433
|
const needle = String(args?.find ?? '').toLowerCase()
|
|
3355
|
-
const rows = needle === '' ?
|
|
3434
|
+
const rows = needle === '' ? ranked : ranked.filter((each) => `${String(each.url)} ${String(each.mime)}`.toLowerCase().includes(needle))
|
|
3356
3435
|
const save = args?.save
|
|
3357
3436
|
if (save === undefined || save === null || save === '') {
|
|
3358
3437
|
if (rows.length === 0) {
|
|
3359
|
-
|
|
3360
|
-
|
|
3361
|
-
|
|
3438
|
+
if (known.length === 0) return 'Nothing to save yet: play the video or sound once, then ask again — the request it makes is the file.'
|
|
3439
|
+
if (needle !== '') return `Nothing matches ${String(args?.find)}; ${String(known.length)} played here, the ones worth keeping are gone.`
|
|
3440
|
+
return `Only ads and beacons played here (${String(known.length)}); all: true shows them.`
|
|
3362
3441
|
}
|
|
3363
|
-
const
|
|
3442
|
+
const hidden = known.length - ranked.length
|
|
3443
|
+
const lines = rows.slice(0, 12).map((each, at) => {
|
|
3444
|
+
const mine = players.find((one) => one.url === each.url || String(one.url).startsWith(String(each.url)) || String(each.url).startsWith(String(one.url)))
|
|
3445
|
+
const mark = at === 0 && mine !== undefined ? 'main ' : ''
|
|
3446
|
+
const length = mine !== undefined && mine.dur > 0 ? ` ${clockOf(mine.dur)}` : ''
|
|
3447
|
+
return `${String(at + 1)}. ${mark}${String(each.kind)}${length} ${sizeText(each.bytes)} · ${String(each.url).slice(0, 160)}`
|
|
3448
|
+
})
|
|
3364
3449
|
if (rows.length > 12) lines.push(`… ${String(rows.length - 12)} more; narrow it with find`)
|
|
3365
|
-
lines.push(
|
|
3450
|
+
if (hidden > 0) lines.push(`… ${String(hidden)} ad or beacon kept out; all: true shows them`)
|
|
3451
|
+
lines.push('save: <row> or main')
|
|
3366
3452
|
return lines.join('\n')
|
|
3367
3453
|
}
|
|
3368
|
-
const
|
|
3454
|
+
const named = typeof save === 'string' && save.toLowerCase() === 'main'
|
|
3455
|
+
const wanted = named ? rows[0] : typeof save === 'number' ? rows[save - 1] : rows.find((each) => String(each.url).includes(String(save)))
|
|
3369
3456
|
if (wanted === undefined) {
|
|
3370
|
-
return `No media row ${String(save)}. ${rows.length === 0 ? 'Nothing to save yet.' : 'Call browser_media without save to see the rows.'}`
|
|
3457
|
+
return `No media row ${String(save)}. ${rows.length === 0 ? 'Nothing to save yet.' : 'Call browser_media without save to see the rows; all: true includes ads.'}`
|
|
3371
3458
|
}
|
|
3372
3459
|
const cookieAnswer = await page.cdp.send('Network.getCookies', { urls: [String(wanted.url)] }).catch(() => ({ cookies: [] }))
|
|
3373
3460
|
const cookie = (cookieAnswer?.cookies ?? []).map((each) => `${String(each.name)}=${String(each.value)}`).join('; ')
|