dsh-inner-browser-use 0.35.3 → 0.37.0

package/README.md

@@ -20,6 +20,7 @@ live picture of that browser, and your own cursor keeps working on the picture.
 | `browser_key` | Presses one key: Enter, Tab, Escape, Backspace, Delete, Insert, arrows, Home, End, PageUp, PageDown, F1-F12, the numeric keypad, letters, digits, punctuation and every shifted symbol, plus ContextMenu, CapsLock, PrintScreen and media keys. `Ctrl`/`Alt`/`Shift`/`Meta`/`Mod` modifiers, chords with `+`, and `Ctrl+C/X/V/A/Z/Y` run the real editing commands. |
 | `browser_scroll` | Scrolls the wheel at the centre of the page, or at `x`/`y`. `dy: 600` is about a screen down. |
 | `browser_shot` | Saves a full-size PNG into `~/.dsh-inner-browser-use/screenshots` and returns the path. |
+| `browser_media` | Lists the video and sound the page already fetched, with sizes, the one its own player shows first and marked `main`; `save: <row>`, `save: main` or part of the address keeps one. Ads and beacons stay out unless `all: true`; `watch: false` stops remembering the page. A stream is followed to its pieces and joined, so nothing has to be installed. |
 | `browser_state` | Reports whether the browser runs, which profile it uses, its address and title, and how many columns are showing it. |
 
 `browser_read` on the same page is the words only, in Markdown (measured: 4,520

package/README.zh.md

@@ -19,6 +19,7 @@ profile，不是产品自带的那个浏览页签：右栏放的是它的画面
 | `browser_key` | 按一个键：Enter、Tab、Escape、Backspace、Delete、上下左右、Home、End、PageUp、PageDown。 |
 | `browser_scroll` | 在页面中心滚轮，也可以指定 `x`/`y`。`dy: 600` 约一屏。 |
 | `browser_shot` | 存一张 PNG 到 `~/.dsh/inner-browser-use/screenshots`，返回路径；`region: [x, y, w, h]` 只截一块（游戏、图表、看局部）。 |
+| `browser_media` | 列出页面已经请求过的视频和声音（带大小），页面自己播放器里那个排第一、标 `main`；`save: <行号>`、`save: main` 或地址片段存下来。广告和统计默认不出现（`all: true` 才看），`watch: false` 关掉这个页面的记录。流会顺着播放列表把片段取回拼好，不用装任何东西。 |
 | `browser_state` | 报浏览器在不在跑、用的哪个 profile、地址、标题，以及有几个右栏在看它。 |
 
 点、输入、按键之后的那一行会多一句结果：`· navigated to …`（换页了）、`· content changed`（页内有变化）、`· no visible change`（点了没动）、Flash 页则是 `· screen updated` / `· no frame change`；`read: true` 时不再多说。

package/docs/ARTIFACTS.md

@@ -8,7 +8,7 @@
 
 - **项目**：会话工作目录的文件夹名，加路径哈希前 6 位，例如 `dsh-inner-browser-use-a1b2c3`。同名文件夹、不同路径不会混。
 - **会话**：会话 id 去掉 `session-` 后的前 8 位，例如 `72557d6e`。同一项目里两个会话各存各的。
-- **种类**：`shots/` 是 `browser_shot` 的截图，`pictures/` 是 `browser_grab` 抓的图，`pdf/` 是 `browser_pdf` 打印的文档。
+- **种类**：`shots/` 是 `browser_shot` 的截图，`pictures/` 是 `browser_grab` 抓的图，`pdf/` 是 `browser_pdf` 打印的文档，`media/` 是 `browser_media` 存下的视频、音频和流。
 
 文件名默认带时间戳（`shot-20261028-124500.png`），给了 `name` 就用它；名字里的分隔符、连续点、开头点都会被去掉，跳不出这个目录。
 
@@ -16,4 +16,18 @@
 
 调用环境里没有项目或会话信息时，落到 `no-project/no-chat/<种类>/`，不猜。
 
+## 整个目录
+
+```
+~/.dsh-inner-browser-use/
+  artifacts/<项目>/<会话>/<种类>/   给人看的产出，只在这里（见上）
+  chrome/                          浏览器自己的数据：登录态、Cookie、缓存
+  flash/                           Flash 播放器运行时，第一次用 flash:true 才装
+  browser.log                     最近一次浏览器启动和页面记录
+  asks.log                        人问过什么、答了什么
+  tabs.json / mirror.json / flash.json / watcher.json   上次开着哪些页
+```
+
+`chrome/` 是浏览器进程自己在读写，不要手动改；要清登录态，用 `browser_open` 的档位或插件自己的入口，别删目录里的某个文件。上面这些都在一个根目录下，卸载插件前想留产出就拷 `artifacts/`。
+
 测试里用 `DSH_INNER_BROWSER_ARTIFACTS` 指向临时目录，不写真实 home。

package/docs/BENCH.md

@@ -1,3 +1,20 @@
+## 只列该看的那一个（0.37.0，同一台本地页面，加了两个广告请求）
+
+页面自己放一个 `<video src="/movie.mp4">`、一个 `<audio>`、fetch 一个 master 播放列表，再 fetch 两条广告（`/ads/preroll.mp4` 9 MB、`/px/beacon.mp4` 2 B）。不筛选时 5 条都在；默认列表剩 3 条，页面播放器那个标 `main` 排第一，`save: main` 存的就是它。
+
+| 项 | 0.36.0 | 0.37.0 |
+|---|---|---|
+| 列表行数（同一页面 5 条请求） | 5 | 3，广告单独一行说数量 |
+| 首行 | `1. video 20 B · /movie.mp4` | `1. main video 156 B · /movie.mp4` |
+| 找人要的那个 | 数行号 / 猜地址 | `save: main` |
+| 列表耗时 | 0 ms（纯内存） | 1 ms（多一次页面查询拿播放器与时长） |
+
+`media-pick` 1 ms，预算 `media-pick: 150 ms`；`all: true` 回到 5 行。`media-sniff` 仍 0 ms（记录仍是网络日志附带的），`media-save` 4 ms、96 B、`6 parts joined`，`budget checks 17 ok`。
+
+## 媒体嗅探与流保存（0.36.0，本地 http 页面，`scripts/bench.mjs`）
+
+页面自己放 `<video>`、`<audio>` 并 fetch 一个 master 播放列表：加载完的那一刻网络记录里已经有 3 条，`media-sniff` 0 ms（没有额外请求，嗅探就是已有的网络记录）。保存 master → 1080 → 6 个片段，`media-save` 5 ms、96 B，报 `6 parts joined`；列表首行 59 字符。预算 `media-sniff: 400 ms`、`media-save: 500 ms` 均通过，`budget checks 16 ok`。
+
 ## Flash 镜像按栏改窗口（0.34.48，Electron Flash 运行时，右栏 747×1196）
 
 Flash 路径原来采集一个固定的 1440×900 窗口，竖高右栏里上下留黑边。现在 `setViewport` 直接改窗口大小，采集源与栏同比例。

package/index.js

@@ -944,7 +944,8 @@ export function apply(ctx) {
         '先接着用已有的页：`browser_view`（拿编号控件与可见文字）→ `browser_click` / `browser_type` → 再 `browser_view` 直到出现要的结果；页面要等就重复读，不 sleep。`browser_open` 只用于没有页、要换新地址、或要明确刷新；同一个地址不要反复重开。',
         '页面很长、只想确认某句话/按钮在哪时用 `browser_find`，别整页读。要读长文（文章、文档、issue、论坛帖）用 `browser_read`：它回 Markdown，只花几十分之一的 token，链接可点；要动手点、填表才用 `browser_view` 拿编号。',
         '要保存页面样子用 `browser_shot`，它会给出 PNG 绝对路径；要交给人看的文件，把绝对路径写进回复。',
-        '游戏、canvas、Flash：画面自己画的地方，`browser_view` 和 `browser_read` 读不出里面是什么。先 `browser_prepare` 拿画面位置、帧率、按键是否到达和下一步；之后看 `browser_shot` 的图定位置，按坐标点、按住、按组合键，再拍一张看画面变了没有——变了就是这一步生效了。当年用 Flash 做的游戏用 `browser_open` 带 `flash:true` 在栏内跑，不装播放器、不开窗口。',
+        '页面放视频或音乐、人想留下它时用 `browser_media`：它列的是页面自己已经请求过的媒体（带大小），页面自己播放器里那个标 `main` 排第一，广告和统计请求默认不出现（`all:true` 才看），`save: main` 存第一个；视频是流的话会顺着播放列表把片段取回拼成整段，不用装 yt-dlp 或 ffmpeg。先让页面播一下再列，播放请求才是文件。不想要这个页面的记录就 `watch:false`。',
+        '游戏、canvas、Flash：画面自己画的地方，`browser_view` 和 `browser_read` 读不出里面是什么。先 `browser_prepare` 拿画面位置、帧率、按键是否到达和下一步；之后看 `browser_shot` 的图定位置，按坐标点、按住、按组合键，再拍一张看画面变了没有——变了就是这一步生效了。当年用 Flash 做的游戏用 `browser_open` 带 `flash:true` 在栏内跑，不装播放器、不开窗口；Flash 默认关，`flash:true` 开、`flash:false` 回到普通 Chrome，两边都只在本会话右栏。',
       ].join('\n'),
     }), 'dsh-inner-browser-use: system prompt section')
   }

package/media.js

@@ -0,0 +1,309 @@
+/**
+ * Media a page plays, and the way to the file behind it.
+ *
+ * yt-dlp solves this by teaching itself every site. The page already knows
+ * better than any list: it asked for the file, and this browser saw the
+ * request. So the sniffing half is free — the network log this plugin already
+ * keeps says which URL is the movie — and the rest is playlist arithmetic:
+ * a master playlist names its variants, a media playlist names its segments,
+ * and the segments are plain HTTP with the cookies the page already has.
+ *
+ * Nothing here touches the disk or the network: parsing and planning are pure,
+ * so they are the part that can be tested without a page.
+ */
+
+/** A mime that means "this is the movie itself", not a page about it. */
+const FILE_MIME = /^(video|audio)\/|^application\/(vnd\.apple\.mpegurl|x-mpegurl|dash\+xml|octet-stream)$/i
+
+/** A URL that ends like a file a player would stream. */
+const FILE_PATH = /\.(m3u8|mpd|mp4|m4v|webm|mkv|mov|flv|ts|m4s|mp3|m4a|aac|ogg|opus|wav)(\?|#|$)/i
+
+/** Same-origin noise that sniffing must not report as a movie. */
+const NOT_MEDIA = /\.(m3u8|mpd|mp4|webm|mov|flv|ts|m4s|mp3|m4a|aac|ogg|opus|wav)\b|^data:|^blob:/i
+
+/**
+ * Is this request something a person would call a video or a song?
+ * @param {string} url - request URL.
+ * @param {string} [mime] - response mime, when known.
+ * @param {string} [type] - CDP resource type, when known.
+ * @returns {boolean} true when it is worth listing.
+ */
+export function isMedia(url, mime = '', type = '') {
+  const at = String(url ?? '')
+  const said = String(mime ?? '').toLowerCase()
+  if (at === '' || at.startsWith('data:') || at.startsWith('blob:')) return false
+  if (said.startsWith('image/') || said.startsWith('font/') || said.startsWith('text/html') || said.includes('javascript') || said.includes('css')) return false
+  if (said === 'application/json' || said.startsWith('application/json')) return false
+  if (FILE_MIME.test(said)) return true
+  if (String(type ?? '') === 'Media') return true
+  return FILE_PATH.test(at) && !/\.(html?|php|aspx?)$/i.test(at)
+}
+
+/**
+ * Which word a person would use for it.
+ * @param {string} url - request URL.
+ * @param {string} [mime] - response mime.
+ * @returns {'video'|'audio'|'stream'|'file'} the kind.
+ */
+export function kindOf(url, mime = '') {
+  const said = String(mime ?? '').toLowerCase()
+  const at = String(url ?? '')
+  if (/mpegurl|dash\+xml/.test(said) || /\.(m3u8|mpd)(\?|#|$)/i.test(at)) return 'stream'
+  if (said.startsWith('audio/')) return 'audio'
+  if (/\.(mp3|m4a|aac|ogg|opus|wav)(\?|#|$)/i.test(at)) return 'audio'
+  if (said.startsWith('video/') || /\.(mp4|m4v|webm|mkv|mov|flv|ts|m4s)(\?|#|$)/i.test(at)) return 'video'
+  return 'file'
+}
+
+/**
+ * Turn the relative parts of a playlist into addresses that can be fetched.
+ * @param {string} base - the playlist's own URL.
+ * @param {string} href - a line from it.
+ * @returns {string} absolute URL, or '' when it is a comment.
+ */
+export function absoluteUrl(base, href) {
+  const line = String(href ?? '').trim()
+  if (line === '' || line.startsWith('#')) return ''
+  try {
+    return new URL(line, base).toString()
+  } catch {
+    return ''
+  }
+}
+
+/**
+ * Read the attributes on one `#EXT-X-STREAM-INF:` line.
+ * @param {string} line - the tag line.
+ * @returns {{bandwidth: number, width: number, height: number, codecs: string}} what it says.
+ */
+export function streamInfo(line) {
+  const text = String(line ?? '')
+  const number = (name) => {
+    const found = new RegExp(`${name}=(\\d+)`, 'i').exec(text)
+    return found === null ? 0 : Number(found[1])
+  }
+  const size = /RESOLUTION=(\d+)x(\d+)/i.exec(text)
+  const codecs = /CODECS="([^"]*)"/i.exec(text)
+  return {
+    bandwidth: number('BANDWIDTH') || number('AVERAGE-BANDWIDTH'),
+    width: size === null ? 0 : Number(size[1]),
+    height: size === null ? 0 : Number(size[2]),
+    codecs: codecs === null ? '' : String(codecs[1]).slice(0, 40),
+  }
+}
+
+/**
+ * A master playlist is a menu of the same movie at different sizes.
+ * @param {string} text - playlist body.
+ * @param {string} base - playlist URL.
+ * @returns {Array<{bandwidth: number, width: number, height: number, codecs: string, url: string}>} variants, best first.
+ */
+export function parseMaster(text, base) {
+  const lines = String(text ?? '').split(/\r?\n/)
+  const variants = []
+  for (let i = 0; i < lines.length; i += 1) {
+    const line = lines[i].trim()
+    if (!line.toUpperCase().startsWith('#EXT-X-STREAM-INF')) continue
+    const info = streamInfo(line)
+    let at = i + 1
+    while (at < lines.length && (lines[at].trim() === '' || lines[at].trim().startsWith('#'))) at += 1
+    const url = at < lines.length ? absoluteUrl(base, lines[at]) : ''
+    if (url === '') continue
+    variants.push({ ...info, url })
+    i = at
+  }
+  variants.sort((left, right) => (right.bandwidth || right.height) - (left.bandwidth || left.height))
+  return variants
+}
+
+/**
+ * A media playlist is the movie cut into pieces.
+ * @param {string} text - playlist body.
+ * @param {string} base - playlist URL.
+ * @returns {{segments: string[], init: string, duration: number, encrypted: boolean}} the pieces and what they need first.
+ */
+export function parsePlaylist(text, base) {
+  const lines = String(text ?? '').split(/\r?\n/)
+  const segments = []
+  let init = ''
+  let duration = 0
+  let encrypted = false
+  for (const raw of lines) {
+    const line = raw.trim()
+    if (line === '') continue
+    if (line.startsWith('#EXTINF:')) duration += Number.parseFloat(line.slice(8)) || 0
+    else if (line.toUpperCase().startsWith('#EXT-X-MAP')) {
+      const found = /URI="([^"]+)"/i.exec(line)
+      if (found !== null) init = absoluteUrl(base, found[1])
+    } else if (line.toUpperCase().startsWith('#EXT-X-KEY')) {
+      if (!/METHOD=NONE/i.test(line)) encrypted = true
+    } else if (!line.startsWith('#')) {
+      const url = absoluteUrl(base, line)
+      if (url !== '') segments.push(url)
+    }
+  }
+  return { segments, init, duration: Math.round(duration), encrypted }
+}
+
+/**
+ * Which playlist this body is.
+ * @param {string} text - playlist body.
+ * @returns {'master'|'media'|null} the kind.
+ */
+export function playlistKind(text) {
+  const body = String(text ?? '')
+  if (!body.trimStart().startsWith('#EXTM3U')) return null
+  if (/#EXT-X-STREAM-INF/i.test(body)) return 'master'
+  return 'media'
+}
+
+/**
+ * The one variant a person would want: tall enough to be worth watching,
+ * cheap enough to fetch. A high one is never the answer to "save this".
+ * @param {Array<{bandwidth: number, width: number, height: number, url: string}>} variants - from parseMaster.
+ * @param {{height?: number, bandwidth?: number}} [wanted] - caps.
+ * @returns {{bandwidth: number, width: number, height: number, url: string}|null} the pick.
+ */
+export function pickVariant(variants, wanted = {}) {
+  const list = (variants ?? []).filter((each) => each !== null && typeof each === 'object' && String(each.url ?? '') !== '')
+  if (list.length === 0) return null
+  const capHeight = Number(wanted.height ?? 0)
+  const capBandwidth = Number(wanted.bandwidth ?? 0)
+  let allowed = list
+  if (capHeight > 0) allowed = list.filter((each) => (each.height || 0) <= capHeight)
+  if (capBandwidth > 0) allowed = allowed.filter((each) => (each.bandwidth || 0) <= capBandwidth)
+  /* A cap nobody fits is not an empty answer: the smallest is still better
+     than nothing. */
+  if (allowed.length === 0) allowed = [...list].sort((left, right) => (left.bandwidth || left.height) - (right.bandwidth || right.height)).slice(0, 1)
+  return allowed.slice().sort((left, right) => (right.bandwidth || right.height) - (left.bandwidth || left.height))[0]
+}
+
+/**
+ * Follow a manifest down to the URLs that hold bytes.
+ * @param {string} url - what the page asked for.
+ * @param {string} body - what came back.
+ * @param {{height?: number, bandwidth?: number}} [wanted] - size cap.
+ * @returns {{kind: 'file'|'hls', parts: string[], init: string, duration: number, variant: object|null, encrypted: boolean}} what to fetch.
+ */
+export function planOf(url, body, wanted = {}) {
+  const kind = playlistKind(body)
+  if (kind === 'master') {
+    const variant = pickVariant(parseMaster(body, url), wanted)
+    return { kind: 'hls', parts: [], init: '', duration: 0, variant, encrypted: false }
+  }
+  if (kind === 'media') {
+    const list = parsePlaylist(body, url)
+    return { kind: 'hls', parts: list.segments, init: list.init, duration: list.duration, variant: null, encrypted: list.encrypted }
+  }
+  return { kind: 'file', parts: [url], init: '', duration: 0, variant: null, encrypted: false }
+}
+
+/**
+ * Fetch many URLs, but only a few at a time: a hundred segments asked for at
+ * once is a hundred sockets and a server that starts refusing.
+ * @template T
+ * @param {string[]} urls - in order.
+ * @param {(url: string) => Promise<T>} get - one URL.
+ * @param {number} [width] - how many at once.
+ * @returns {Promise<T[]>} in the same order as the input.
+ */
+export async function gather(urls, get, width = 6) {
+  const out = new Array(urls.length)
+  let next = 0
+  const take = async () => {
+    for (;;) {
+      const at = next
+      next += 1
+      if (at >= urls.length) return
+      out[at] = await get(urls[at])
+    }
+  }
+  const taking = []
+  for (let i = 0; i < Math.max(1, Math.min(width, urls.length)); i += 1) taking.push(take())
+  await Promise.all(taking)
+  return out
+}
+
+/**
+ * How a person would see the size of a pile of bytes.
+ * @param {number} bytes - byte count.
+ * @returns {string} "12 MB" and the like.
+ */
+export function sizeText(bytes) {
+  const n = Number(bytes ?? 0)
+  if (!(n > 0)) return '?'
+  if (n >= 1_000_000_000) return `${(n / 1_000_000_000).toFixed(1)} GB`
+  if (n >= 1_000_000) return `${(n / 1_000_000).toFixed(1)} MB`
+  if (n >= 1000) return `${Math.round(n / 1000)} kB`
+  return `${String(n)} B`
+}
+
+/**
+ * Ad servers and ad slots, recognised by themselves rather than by size: an ad
+ * is often the biggest file on the page, and a real clip can be small.
+ */
+const AD_HOST = /^(?:(?:[\w-]+\.)*(?:doubleclick\.net|googlesyndication\.com|googleadservices\.com|amazon-adsystem\.com|adsrvr\.org|taboola\.com|outbrain\.com|criteo\.(?:net|com)|scorecardresearch\.com|moatads\.com|adnxs\.com|pubmatic\.com|rubiconproject\.com|openx\.net)$)$/i;
+const AD_SLOT = /^(?:ad|ads|adv|adserver|advert(?:isement|ising)?|pre-?roll|mid-?roll|post-?roll|banner|sponsor|promo|tracking|tracker|pixel|beacon|analytics|telemetry)\d*$/i
+
+/**
+ * Whether an address is an ad or a measurement call.
+ * @param {string} url - the address.
+ * @returns {boolean} true when it is worth hiding from a person picking a file.
+ */
+export function adLike(url) {
+  let parsed
+  try {
+    parsed = new URL(String(url))
+  } catch {
+    return false
+  }
+  if (AD_HOST.test(parsed.hostname)) return true
+  const pieces = parsed.pathname.split('/').filter((piece) => piece !== '')
+  const named = pieces.slice(0, -1).some((piece) => AD_SLOT.test(piece))
+  if (named) return true
+  const file = pieces.length === 0 ? '' : pieces[pieces.length - 1]
+  return /^(?:ad|ads|adv|preroll|banner|pixel|beacon|track|tracking)[-_.]/i.test(file)
+}
+
+/**
+ * How much a row looks like the thing a person came for: the player on the page
+ * first, then a real file, and ads or beacons last.
+ * @param {{url: string, mime?: string, kind?: string, bytes?: number}} row - one media request.
+ * @param {Array<{url: string, dur?: number}>} [players] - what the page's own `<video>` / `<audio>` plays.
+ * @returns {number} the score; higher is more likely the point of the page.
+ */
+export function coreScore(row, players = []) {
+  const url = String(row?.url ?? '')
+  const bytes = Number(row?.bytes ?? 0)
+  const kind = String(row?.kind ?? '')
+  const mime = String(row?.mime ?? '')
+  let score = 0
+  const mine = players.some((one) => {
+    const said = String(one?.url ?? '')
+    return said !== '' && (said === url || url.startsWith(said) || said.startsWith(url))
+  })
+  if (mine) score += 100
+  if (adLike(url)) score -= 80
+  if (bytes >= 1_000_000) score += 25
+  else if (bytes >= 200_000) score += 10
+  else if (bytes > 0 && bytes < 20_000 && kind !== 'stream') score -= 20
+  if (kind === 'stream') score += 15
+  if (/^video\//i.test(mime)) score += 5
+  if (/^audio\//i.test(mime)) score += 3
+  return score
+}
+
+/**
+ * The rows in the order a person would want them, with ads gone unless asked.
+ * @param {Array<object>} rows - media rows, any order.
+ * @param {Array<{url: string, dur?: number}>} [players] - the page's own players.
+ * @param {boolean} [all] - keep the ads and the beacons too.
+ * @returns {Array<object>} the rows to show.
+ */
+export function coreRows(rows, players = [], all = false) {
+  const ranked = rows.map((row) => ({ row, score: coreScore(row, players) }))
+  return ranked
+    .filter((one) => all || one.score >= 0)
+    .sort((left, right) => right.score - left.score || Number(right.row?.seq ?? 0) - Number(left.row?.seq ?? 0))
+    .map((one) => one.row)
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-inner-browser-use",
-  "version": "0.35.3",
+  "version": "0.37.0",
   "description": "An agent-owned Chromium sidecar mirrored into the DeepSeek Harness right column: the agent drives it, the person watches and can take the mouse, and logins survive restarts.",
   "type": "module",
   "main": "index.js",
@@ -17,6 +17,7 @@
     "tools.js",
     "read.js",
     "review.js",
+    "media.js",
     "client.js",
     "locale/*.json",
     "cordis.patch.yml",
@@ -48,7 +49,7 @@
     "dsh": ">=0.2.0"
   },
   "scripts": {
-    "test": "node --test test/artifacts.test.mjs test/client.test.mjs test/cookies.test.mjs test/diff.test.mjs test/fresh.test.mjs test/host.test.mjs test/input.test.mjs test/isolation.test.mjs test/keyboard.test.mjs test/leak.test.mjs test/login.test.mjs test/no-duplicate.test.mjs test/package.test.mjs test/readfault.test.mjs test/refs.test.mjs test/review.test.mjs test/session.test.mjs test/settle.test.mjs test/sidecar.test.mjs test/swf.test.mjs test/tabs.test.mjs test/trouble.test.mjs",
+    "test": "node --test test/artifacts.test.mjs test/client.test.mjs test/cookies.test.mjs test/diff.test.mjs test/fresh.test.mjs test/host.test.mjs test/input.test.mjs test/isolation.test.mjs test/keyboard.test.mjs test/leak.test.mjs test/login.test.mjs test/media.test.mjs test/no-duplicate.test.mjs test/package.test.mjs test/readfault.test.mjs test/refs.test.mjs test/review.test.mjs test/session.test.mjs test/settle.test.mjs test/sidecar.test.mjs test/swf.test.mjs test/tabs.test.mjs test/trouble.test.mjs",
     "doctor": "node scripts/doctor.mjs",
     "breadth": "node scripts/breadth.mjs",
     "live": "node scripts/live.mjs",

package/sidecar.js

@@ -11,6 +11,7 @@ import { homedir } from 'node:os'
 import { join } from 'node:path'
 import { fileURLToPath } from 'node:url'
 import { readyFlash } from './runtime/flash-setup.mjs'
+import { isMedia, kindOf } from './media.js?m=1'
 
 /**
  * Windows keeps a browser under Program Files (machine-wide) or LocalAppData
@@ -575,6 +576,10 @@ export class SidecarPage {
        be understood without a screenshot or a retry. */
     this.network = []
     this.requests = new Map()
+    /* The movie a page plays is one of the requests it already made, so the
+       file behind it costs no second look: the network log is the sniff. */
+    this.media = new Map()
+    this.mediaOn = true
     /* A PDF is a document the mirror cannot draw: Chrome leaves the page blank
        and a model looking at it would think the address was empty. What it is
        and where the bytes went is state, not a guess. */
@@ -713,6 +718,7 @@ export class SidecarPage {
          it navigates the same tab to its own page, and the PDF is still what
          the tab shows. */
       if (String(params.type ?? '') === 'Document' && !String(request.url ?? '').startsWith('chrome-extension://')) this.pdf = null
+      if (String(params.type ?? '') === 'Document' && !String(request.url ?? '').startsWith('chrome-extension://')) this.media = new Map()
       this.requests.set(id, {
         method: String(request.method ?? 'GET'),
         url: String(request.url ?? '').slice(0, 300), // ui-copy-ignore
@@ -732,6 +738,8 @@ export class SidecarPage {
           bytes: Number(params.response?.encodedDataLength ?? 0),
         }
       }
+      if (known !== undefined) known.status = Number(params.response?.status ?? 0)
+      this.noteMedia(params)
       if (known === undefined) return
       known.status = Number(params.response?.status ?? 0)
       this.troubleSeq += 1
@@ -749,6 +757,56 @@ export class SidecarPage {
     }
   }
 
+  /**
+   * Keep the requests that hold a movie or a song, so asking "what can be saved
+   * here" is a memory read instead of a network sweep.
+   * @param {{requestId?: string, type?: string, response?: object}} params - a Network.responseReceived payload.
+   * @returns {void} nothing.
+   */
+  noteMedia(params) {
+    if (this.mediaOn === false) return
+    const response = params?.response ?? {}
+    const url = String(response.url ?? '')
+    const mime = String(response.mimeType ?? '')
+    if (!isMedia(url, mime, String(params?.type ?? ''))) return
+    const id = String(params?.requestId ?? '')
+    if (id === '') return
+    const headers = response.headers ?? {}
+    const length = Number(headers['content-length'] ?? headers['Content-Length'] ?? response.encodedDataLength ?? 0)
+    const existing = this.media.get(id)
+    this.media.set(id, {
+      url,
+      mime,
+      kind: kindOf(url, mime),
+      bytes: Number.isFinite(length) ? length : 0,
+      status: Number(response.status ?? 0),
+      seq: existing?.seq ?? this.troubleSeq,
+    })
+    if (this.media.size > 60) this.media.delete(this.media.keys().next().value)
+  }
+
+  /**
+   * Stop or resume remembering media on this page; stopping empties the list so
+   * nothing is kept from before the switch.
+   * @param {boolean} on - false stops, true resumes.
+   * @returns {boolean} what it is set to.
+   */
+  mediaWatch(on) {
+    this.mediaOn = on !== false
+    if (!this.mediaOn) this.media = new Map()
+    return this.mediaOn
+  }
+
+  /**
+   * What on this page is worth saving, newest first.
+   * @returns {Array<{url: string, mime: string, kind: string, bytes: number, status: number}>} the media.
+   */
+  mediaList() {
+    const byUrl = new Map()
+    for (const each of this.media.values()) byUrl.set(each.url, each)
+    return [...byUrl.values()].sort((left, right) => right.seq - left.seq)
+  }
+
   /**
    * Follow a tab the page just opened: the mirror shows the new page and the
    * popup is closed, so there is still one page to drive.

package/tools.js

@@ -14,10 +14,47 @@ import { createHash } from 'node:crypto'
 import { homedir } from 'node:os'
 import { basename, join } from 'node:path'
 import { mkdir, readdir, stat, unlink, writeFile } from 'node:fs/promises'
-import { VIEW } from './sidecar.js?v=1'
+import { VIEW, DESKTOP_UA } from './sidecar.js?v=1'
+import { planOf, gather, sizeText, coreRows } from './media.js?m=2'
 import { readPage, renderPage, readArticle, readRelevant, findOnPage, locate, locateFirst, FRAME_BASE, covering, contextFor, identityOf, sameThing, differs, labelSpot, selectSpot, bounds, pictures, clickInDom, writeInDom, REF_GONE } from './read.js?v=1'
 import { brief as askBrief, recordRound, lastRoundNote, lastRound, markRound, MARK_STATUS } from './review.js?v=1'
 
+/**
+ * What a saved file should be called, from the address when it says so and the
+ * type when it does not.
+ * @param {string} url - the media's address.
+ * @param {string} [mime] - what the server called it.
+ * @returns {string} an extension with the dot.
+ */
+function extensionOf(url, mime = '') {
+  const known = /\.(mp4|m4v|webm|mkv|mov|flv|ts|m4s|mp3|m4a|aac|ogg|opus|wav)(\?|#|$)/i.exec(String(url ?? ''))
+  if (known !== null) return `.${known[1].toLowerCase()}`
+  const said = String(mime ?? '').toLowerCase()
+  if (said.includes('mp4') || said.includes('m4v')) return '.mp4'
+  if (said.includes('webm')) return '.webm'
+  if (said.includes('mpegurl')) return '.ts'
+  if (said.startsWith('audio/')) return '.m4a'
+  return '.bin'
+}
+
+/** What the page's own players are showing, in one look: the address, the length, whether it runs. */
+async function playersOf(page) {
+  const answer = await page.cdp.send('Runtime.evaluate', {
+    expression: `(() => [...document.querySelectorAll('video,audio')].map((one) => ({ url: one.currentSrc || one.src || '', playing: !one.paused, dur: Number.isFinite(one.duration) ? Math.round(one.duration) : 0 })))()`,
+    returnByValue: true,
+  }).catch(() => null)
+  const seen = answer?.result?.value
+  return Array.isArray(seen) ? seen.filter((one) => String(one?.url ?? '') !== '') : []
+}
+
+/** Seconds as a person writes them: 2:13, 1:02:00. */
+function clockOf(seconds) {
+  const whole = Math.max(0, Math.round(Number(seconds) || 0))
+  const rest = whole % 3600
+  const pad = (value) => String(value).padStart(2, '0')
+  return whole >= 3600 ? `${String(Math.floor(whole / 3600))}:${pad(Math.floor(rest / 60))}:${pad(rest % 60)}` : `${String(Math.floor(whole / 60))}:${pad(rest % 60)}`
+}
+
 /** Which chat the running call belongs to, so each gets its own tab. */
 export const sessionScope = new AsyncLocalStorage()
 
@@ -3313,6 +3350,108 @@ export function sidecarTools(sidecar, refs, host = {}) { // ui-copy-ignore
       }),
     },
 
+    {
+      name: 'browser_media', // ui-copy-ignore
+      description: 'What this page played, and how to keep it: lists the video and sound requests the page already made, the one its own player shows first, with sizes; save one by row number, by part of its address, or save: main. Ads and measurement beacons stay out of the list unless all: true. A stream playlist is followed to its pieces and joined, so nothing has to be installed. watch: false stops remembering this page.',
+      parameters: {
+        type: 'object',
+        properties: {
+          find: { type: 'string', description: 'Only rows whose address contains this.' },
+          save: { type: ['integer', 'string'], description: 'Row number from the list, part of its address, or main for the first row.' },
+          all: { type: 'boolean', description: 'Include ads and beacons too (default false).' },
+          watch: { type: 'boolean', description: 'false stops remembering what this page plays and empties the list; true resumes.' },
+          height: { type: 'number', description: 'Tallest version to take when one address offers several (default 1080).' },
+          name: { type: 'string', description: 'File name without extension.' },
+        },
+        additionalProperties: false,
+      },
+      output: TEXT_OUTPUT, // ui-copy-ignore
+      timeoutMs: 180_000,
+      isConcurrencySafe: () => false,
+      /** @returns {Promise<{text: string}>} the list, or where the file went. */
+      execute: async (args) => withRawPage(async (page) => {
+        if (typeof args?.watch === 'boolean') {
+          const on = typeof page.mediaWatch === 'function' ? page.mediaWatch(args.watch) : false
+          if (!on) return 'Off: this page keeps nothing from now on. watch: true puts it back.'
+          if (args.save === undefined || args.save === null || args.save === '') return 'On: ask again once it has played something.'
+        }
+        const known = typeof page.mediaList === 'function' ? page.mediaList() : []
+        const players = known.length === 0 ? [] : await playersOf(page)
+        const ranked = coreRows(known, players, args?.all === true)
+        const needle = String(args?.find ?? '').toLowerCase()
+        const rows = needle === '' ? ranked : ranked.filter((each) => `${String(each.url)} ${String(each.mime)}`.toLowerCase().includes(needle))
+        const save = args?.save
+        if (save === undefined || save === null || save === '') {
+          if (rows.length === 0) {
+            if (known.length === 0) return 'Nothing to save yet: play the video or sound once, then ask again — the request it makes is the file.'
+            if (needle !== '') return `Nothing matches ${String(args?.find)}; ${String(known.length)} played here, the ones worth keeping are gone.`
+            return `Only ads and beacons played here (${String(known.length)}); all: true shows them.`
+          }
+          const hidden = known.length - ranked.length
+          const lines = rows.slice(0, 12).map((each, at) => {
+            const mine = players.find((one) => one.url === each.url || String(one.url).startsWith(String(each.url)) || String(each.url).startsWith(String(one.url)))
+            const mark = at === 0 && mine !== undefined ? 'main ' : ''
+            const length = mine !== undefined && mine.dur > 0 ? ` ${clockOf(mine.dur)}` : ''
+            return `${String(at + 1)}. ${mark}${String(each.kind)}${length} ${sizeText(each.bytes)} · ${String(each.url).slice(0, 160)}`
+          })
+          if (rows.length > 12) lines.push(`… ${String(rows.length - 12)} more; narrow it with find`)
+          if (hidden > 0) lines.push(`… ${String(hidden)} ad or beacon kept out; all: true shows them`)
+          lines.push('save: <row> or main')
+          return lines.join('\n')
+        }
+        const named = typeof save === 'string' && save.toLowerCase() === 'main'
+        const wanted = named ? rows[0] : typeof save === 'number' ? rows[save - 1] : rows.find((each) => String(each.url).includes(String(save)))
+        if (wanted === undefined) {
+          return `No media row ${String(save)}. ${rows.length === 0 ? 'Nothing to save yet.' : 'Call browser_media without save to see the rows; all: true includes ads.'}`
+        }
+        const cookieAnswer = await page.cdp.send('Network.getCookies', { urls: [String(wanted.url)] }).catch(() => ({ cookies: [] }))
+        const cookie = (cookieAnswer?.cookies ?? []).map((each) => `${String(each.name)}=${String(each.value)}`).join('; ')
+        const headers = { 'user-agent': DESKTOP_UA, accept: '*/*' }
+        if (cookie !== '') headers.cookie = cookie
+        if (/^https?:/i.test(String(page.url ?? ''))) headers.referer = String(page.url)
+        const started = Date.now()
+        const getBinary = async (address) => {
+          const answer = await fetch(address, { headers, redirect: 'follow', signal: AbortSignal.timeout(60_000) })
+          if (!answer.ok) throw new Error(`${String(answer.status)} for ${address.slice(0, 120)}`)
+          return Buffer.from(await answer.arrayBuffer())
+        }
+        const cap = Number(args?.height ?? 1080)
+        const wantedSize = cap > 0 ? { height: cap } : {}
+        let parts
+        let ext = extensionOf(wanted.url, wanted.mime)
+        try {
+          let body = await getBinary(String(wanted.url))
+          parts = [body]
+          const playlist = /\.m3u8(\?|#|$)/i.test(String(wanted.url)) || /mpegurl/i.test(String(wanted.mime ?? ''))
+          if (playlist) {
+            let plan = null
+            let from = String(wanted.url)
+            for (let hop = 0; hop < 2; hop += 1) {
+              plan = planOf(from, body.toString('utf8'), wantedSize)
+              if (plan.kind !== 'hls' || plan.variant === null) break
+              from = plan.variant.url
+              body = await getBinary(from)
+            }
+            if (plan.encrypted === true) return 'That stream is encrypted: the browser can play it, but the pieces are not a file to keep.'
+            if (plan.parts.length === 0) return 'That playlist names no pieces. Play the movie first, then ask again.'
+            const init = plan.init === '' ? [] : [await getBinary(plan.init)]
+            parts = [...init, ...(await gather(plan.parts, getBinary, 6))]
+            ext = plan.init !== '' ? '.mp4' : (/\.ts(\?|#|$)/i.test(plan.parts[0]) ? '.ts' : '.m4s')
+          }
+        } catch (error) {
+          return `Could not fetch it: ${String(error?.message ?? error)}`
+        }
+        const body = Buffer.concat(parts)
+        const dir = artifactDir('media')
+        await mkdir(dir, { recursive: true })
+        const stamp = new Date().toISOString().replace(/[-:]/g, '').replace('T', '-').slice(0, 15)
+        const safe = String(args?.name ?? '').replace(/[^\w.-]/g, '').replace(/\.{2,}/g, '.').replace(/^\.+/, '')
+        const file = join(dir, `${safe === '' ? `media-${stamp}` : safe}${ext}`)
+        await writeFile(file, body)
+        return `${file} — ${sizeText(body.length)} in ${String(Date.now() - started)} ms${parts.length > 1 ? ` (${String(parts.length)} parts joined)` : ''}`
+      }),
+    },
+
     {
       name: 'browser_pdf', // ui-copy-ignore
       description: 'Print the page to a PDF and return its path. Useful when the person wants the document rather than a picture of it.',