dsh-inner-browser-use 0.40.0 → 0.42.0

package/README.md

@@ -12,7 +12,7 @@ live picture of that browser, and your own cursor keeps working on the picture.
 | --- | --- |
 | `browser_open` | Starts the browser (or reuses the running one) and opens an address. `headless: false` shows a real window on the desktop instead. |
 | `browser_view` | Returns the address, the title, every control as `1 … 120`, and the visible text. |
-| `browser_read` | Reads the page the way a person does: one Markdown block of headings, paragraphs, lists and links, no controls. Far fewer tokens than a snapshot on long pages, and the links stay usable. Give it a `query` and only the sentences that answer it come back: BM25 over Readability-style blocks, then per-sentence scoring with heading, position and link-density priors, idf-weighted coverage, MMR de-duplication and a character budget — 92% fewer tokens on a long page in the bench. |
+| `browser_read` | Reads the page the way a person does: one Markdown block of headings, paragraphs, lists and links, no controls. Far fewer tokens than a snapshot on long pages, and the links stay usable. Give it a `query` and only the sentences that answer it come back: BM25 over Readability-style blocks, then per-sentence scoring with heading, position and link-density priors, idf-weighted coverage, MMR de-duplication and a character budget — 92% fewer tokens on a long page in the bench. A question the page cannot answer returns the nearest passages and says so instead of pasting the page (79% fewer tokens in the bench). When the tab has no answer, the other tabs this chat holds are read in the same call and the answer comes back with the tab it came from — one read instead of switch-and-read-again (90% fewer characters in the bench); `across: false` stays on this tab. |
 | `browser_find` | Finds words (or a regex) anywhere on the page: matching controls come back with a clickable number, matching text with its point, each saying whether it is on screen. |
 | `browser_extract` | Returns the items a CSS selector matches, one compact line each with its link, instead of the whole page. |
 | `browser_click` | Clicks a numbered control, or an `x`/`y` point. `double: true` double-clicks. A cookie banner or modal over the control is cleared first, then the click happens. |

package/README.zh.md

@@ -11,7 +11,7 @@ profile，不是产品自带的那个浏览页签：右栏放的是它的画面
 | --- | --- |
 | `browser_open` | 启动浏览器（已在跑就复用）并打开一个地址。传 `headless: false` 会改为在桌面上开真实窗口。 |
 | `browser_view` | 返回地址、标题、编号成 `1 … 120` 的控件，以及可见文字。 |
-| `browser_read` | 按人读文章的方式给 Markdown（标题、段落、列表、链接）。带 `query` 只回能回答的那几句（段落排序后再按句子打分，标题、位置、链接密度做先验，重复句去掉，按字符预算装箱）；问不到时自动改为整页，不再空手回来。页面和上次读的没变就只回一行，不重发正文（`force: true` 强制重发）。 |
+| `browser_read` | 按人读文章的方式给 Markdown（标题、段落、列表、链接）。带 `query` 只回能回答的那几句（段落排序后再按句子打分，标题、位置、链接密度做先验，重复句去掉，按字符预算装箱）；问不到时只回最近的三段并明说没答上，不再把整页端回来；当前页答不上来还会顺手读这个会话自己开着的其它页签，答案在哪页就从哪页回并标出页签标题（bench 里少 90% 字符），`across: false` 只读当前页。页面和上次读的没变就只回一行，不重发正文（`force: true` 强制重发）。 |
 | `browser_extract` | 按 CSS 选择器把列表项读成一行一条、带链接，比整页读省得多。 |
 | `browser_view` 的 `map` | 大页面先看地标：各类控件各多少个、去重后的控件和编号、正文多少字，但不带回正文。 |
 | `browser_click` | 点编号控件，或点 `x`/`y` 坐标。`double: true` 双击。控件被横幅或弹层挡着时，先关掉它再点。 |

package/docs/ARCHITECTURE.md

@@ -186,6 +186,28 @@ sentences are packed to a character budget and restored to document order. Terms
 
 `sentences: false` keeps the paragraph path for callers who want the whole section.
 
+A question the page cannot answer costs the miss, not the page: when no sentence clears the
+cut, `readRelevant` returns the three blocks that ranked highest (each trimmed towards 260
+characters) and no text, and the tool turns that into a line saying nothing directly answers the
+question plus those passages. The whole page is still one `browser_view` away, so the expensive
+fallback — a page pasted into the context for a question it never addressed — is gone.
+
+## Answering across tabs
+
+A question can be answered by a tab that is not the one on screen. Moving the mirror there and
+reading again costs two calls, a screencast teardown, and the person's view moved for nothing, so
+a miss tries the tabs instead: `elsewhere(session)` lists the tabs the calling chat holds (its own
+mirror excluded), opens one read-only CDP handle per tab — `Cdp` on that tab's own
+`webSocketUrl`, never a full `SidecarPage`, never `cast()`, never a screencast stop — and
+`readTabs` pools blocks from all of them under one `maxChars` ceiling and runs the same
+`evidence()` pass once, so a merged answer costs the same characters as a single-tab one. Every
+row keeps the title of the tab its text came from, and every handle is closed in a `finally`.
+
+Only the calling session's claimed tabs are candidates: another chat's tab, and an unclaimed one,
+are not read. At most three extra tabs are opened, and the path only runs when the current tab has
+no answer, so a normal question pays nothing for it. `across: false` turns it off. `pruneStrays`
+counts claimed tabs as owned, because a tab a chat holds is not a stray.
+
 ## Sign-ins and reclaim
 
 Sign-in state lives in the profile, and session cookies — the kind a sign-in usually sets,

package/docs/BENCH.md

@@ -1,3 +1,34 @@
+## 答案在另一个页签（0.42.0）
+
+两个页签：前台一页在讲别的事，后台另一页才有答案。左边是旧做法——切过去、再读一遍；
+右边是一次跨页读。
+
+| 项 | 切页签读两遍 | 一次跨页读 |
+| --- | --- | --- |
+| 调用次数 | 2 | 1 |
+| 回来的字符数 | 879 | 90 |
+| 省下 | — | 90% |
+| 出处 | 要自己记 | 每段标页签标题 |
+| 读这一趟 | 7 ms | 6 ms |
+
+跨页只在当前页答不上来时发生，正常命中路径的延迟不变（`read-answer` 仍是 1172 → 90 字）。
+
+## 问了但页面没答（0.41.0）
+
+同一张故事页，问一个页面里根本没有的词（`quantum chromodynamics`）。左边是旧的兜底：
+把整页当答案交出去；右边是现在只给最近的三段加一句说明。
+
+| 项 | 整页兜底 | 最近几段 |
+| --- | --- | --- |
+| 回来的字符数 | 2089 | 438 |
+| 折合 token | 614 | 129 |
+| 省下 | — | 79% |
+| 最近几段 | — | 3 段（每段 ≤ 260 字） |
+| 是否明说没答上 | 否 | 是 |
+| 读这一趟 | — | 7 ms |
+
+命中路径的 `read-answer` 行不受影响（1172 → 90 字，省 92%）。
+
 ## 只读答案那几句（0.40.0）
 
 同一张本地故事页：24 段会议记录里夹着一段账单说明，问题是「退款多久到账」。一条路按段

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-inner-browser-use",
-  "version": "0.40.0",
+  "version": "0.42.0",
   "description": "An agent-owned Chromium sidecar mirrored into the DeepSeek Harness right column: the agent drives it, the person watches and can take the mouse, and logins survive restarts.",
   "type": "module",
   "main": "index.js",

package/read.js

@@ -1206,6 +1206,17 @@ function fold(word) {
 /** How often a term may occur before it stops adding meaning. */
 const SATURATION = 1.5
 
+/**
+ * A passage short enough to show when nothing answers: one sentence or so,
+ * whitespace collapsed, with an ellipsis saying it was cut.
+ * @param {unknown} value - block text.
+ * @returns {string} the passage.
+ */
+function brief(value) {
+  const body = String(value ?? '').replace(/\s+/g, ' ').trim()
+  return body.length > 260 ? `${body.slice(0, 260).replace(/\s+\S*$/, '')}…` : body
+}
+
 /**
  * Rank blocks against a query the way a search engine ranks a passage: BM25,
  * with the heading counted as a place a term can match. Then read only the
@@ -1383,13 +1394,20 @@ export function evidence(blocks, query, options = {}) {
  */
 export async function readRelevant(page, query, options = {}) {
   const budget = Math.max(400, Math.min(24_000, Number(options.maxChars ?? 4000)))
+  /* A short, untrimmed paragraph is what a miss shows: enough to say what the
+     page is about, never enough to be the page. */
+  const near = (blocks) => blocks
+    .map((block) => brief(block?.t))
+    .filter((body) => body !== '')
+    .slice(0, 3)
+    .map((body) => ({ heading: '', text: body }))
   try {
     const raw = String((await evaluateIn(page, blocks(budget * 8))) ?? '{}')
     const said = JSON.parse(raw)
     const all = Array.isArray(said?.blocks) ? said.blocks : []
     const ranked = rank(all, query)
     if (ranked.length === 0) {
-      return { title: String(said?.title ?? page.title ?? ''), url: String(page.url ?? ''), text: '', chars: 0, matched: 0, total: all.length }
+      return { title: String(said?.title ?? page.title ?? ''), url: String(page.url ?? ''), text: '', chars: 0, matched: 0, total: all.length, nearby: near(all) }
     }
     /* Passages cost whole paragraphs; sentences cost only the ones that say
        something about the query, which on a long page is most of the saving. */
@@ -1413,10 +1431,15 @@ export async function readRelevant(page, query, options = {}) {
       matched += 1
       if (used >= budget) break
     }
+    /* A question the page does not answer should not cost the page: the few
+       blocks that came closest say what the page is about, and the caller can
+       still ask for everything. */
+    const nearby = matched === 0 ? near(ranked.slice(0, 3).map((row) => row.block)) : []
     return {
       title: String(said?.title ?? page.title ?? ''),
       url: String(page.url ?? ''),
       text: lines.join('\n\n'),
+      nearby,
       chars: used,
       matched,
       total: all.length,
@@ -1426,6 +1449,56 @@ export async function readRelevant(page, query, options = {}) {
   }
 }
 
+/**
+ * Answer one question from several tabs at once: each tab is ranked on its own
+ * words, then the sentences are pooled and packed under one ceiling, so an
+ * answer that lives in a background tab costs one read instead of a tab switch
+ * and a second read.
+ * @param {Array<{title?: string, url?: string, cdp: any}>} pages - read-only handles.
+ * @param {string} query - what the caller wants to know.
+ * @param {{maxChars?: number, limit?: number}} [options] - ceiling across all tabs.
+ * @returns {Promise<{rows: Array<{text: string, heading: string, at: number, tab: string}>, tabs: Array<{title: string, url: string, blocks: number, fault?: string}>, chars: number, matched: number, total: number, nearby: Array<{tab: string, text: string}>}>} pooled evidence.
+ */
+export async function readTabs(pages, query, options = {}) {
+  const ceiling = Math.max(400, Math.min(24_000, Number(options.maxChars ?? 4000)))
+  const list = Array.isArray(pages) ? pages.filter((page) => page !== null && page !== undefined && page.cdp !== undefined) : []
+  const share = Math.max(400, Math.floor(ceiling / Math.max(1, list.length)))
+  const tabs = []
+  const pool = []
+  const owner = []
+  for (let at = 0; at < list.length; at += 1) {
+    const page = list[at]
+    const title = String(page.title ?? '') === '' ? String(page.url ?? '') : String(page.title)
+    try {
+      const raw = String((await evaluateIn(page, blocks(share * 8))) ?? '{}')
+      const said = JSON.parse(raw)
+      const found = Array.isArray(said?.blocks) ? said.blocks : []
+      for (const block of found) { pool.push(block); owner.push(at) }
+      tabs.push({ title, url: String(page.url ?? ''), blocks: found.length })
+    } catch (error) {
+      /* One tab that will not answer is not a failed question: the others are
+         still worth reading, and the caller is told which one stayed quiet. */
+      tabs.push({ title, url: String(page.url ?? ''), blocks: 0, fault: String(error?.message ?? error).split('\n')[0].slice(0, 160) })
+    }
+  }
+  const limit = Math.max(1, Math.min(60, Number(options.limit ?? 12)))
+  const found = evidence(pool, query, { maxChars: ceiling, limit })
+  const rows = found.rows.map((row) => {
+    const tab = tabs[owner[row.at] ?? 0]?.title ?? ''
+    return Object.assign({}, row, { tab })
+  })
+  /* Nothing answered anywhere: the nearest passage of every tab says what each
+     one is about, which is what makes the next look cheap. */
+  const nearby = rows.length > 0 ? [] : tabs
+    .map((tab, at) => {
+      const first = pool.find((block, slot) => owner[slot] === at && brief(block?.t) !== '')
+      return first === undefined ? null : { tab: tab.title, text: brief(first.t) }
+    })
+    .filter((row) => row !== null)
+  const chars = rows.reduce((sum, row) => sum + row.text.length, 0)
+  return { rows, tabs, chars, matched: rows.length, total: pool.length, nearby }
+}
+
 /**
  * Read the page as Markdown, through the same deadline as a snapshot.
  * @param {any} page - sidecar page.

package/sidecar.js

@@ -2337,6 +2337,11 @@ export class Sidecar {
       const id = String(page?.targetId ?? '')
       if (id !== '') owned.add(id)
     }
+    /* A tab a chat claimed is a tab that chat holds, even when the mirror is on
+       another one: pruning it would take away the tab a cross-tab read needs. */
+    for (const set of this.claims.values()) {
+      for (const id of set) owned.add(String(id))
+    }
     let remembered = []
     try {
       remembered = Object.values(JSON.parse(readFileSync(tabsFile, 'utf8')) ?? {}).map(String)
@@ -2873,6 +2878,46 @@ export class Sidecar {
     return shown
   }
 
+  /**
+   * Read-only links to the other tabs this chat holds, so one question can be
+   * answered by the tab that has the answer. A handle can read (it has a
+   * connection) and must be closed; it never starts a screencast, moves the
+   * mirror or replaces the page this chat is driving.
+   * @param {string} session - chat asking.
+   * @param {number} [limit] - how many other tabs at most, 1 to 3.
+   * @returns {Promise<Array<{title: string, url: string, cdp: any, close: () => void}>>} handles.
+   */
+  async elsewhere(session, limit = 3) {
+    const most = Math.max(1, Math.min(3, Number(limit) || 3))
+    const shown = await this.windowsFor(session)
+    const rows = await this.tabs(session)
+    /* The chat-level tab list drops the connection address, so the browser's own
+       list is what a read-only handle is opened from. */
+    const source = this.pageFor(String(session ?? '')) ?? this.page
+    const raw = typeof source?.tabs === 'function' ? await source.tabs() : []
+    const byId = new Map(raw.map((tab) => [String(tab.targetId), tab]))
+    const out = []
+    for (const row of shown) {
+      if (out.length >= most) break
+      if (row.current === true) continue
+      const tab = byId.get(String(rows[row.index]?.targetId ?? ''))
+      if (tab === undefined || String(tab.webSocketUrl ?? '') === '') continue
+      try {
+        const cdp = new Cdp(tab.webSocketUrl)
+        await cdp.ready
+        out.push({
+          title: String(row.title === '' ? tab.title : row.title),
+          url: String(row.url === '' ? tab.url : row.url),
+          cdp,
+          close: () => cdp.close(),
+        })
+      } catch {
+        /* A tab that will not open is one tab less; the read still happens. */
+      }
+    }
+    return out
+  }
+
   /**
    * Move the mirror to another tab, leaving the old one open where it was.
    * @param {number} index - a row from `tabs()`.

package/tools.js

@@ -17,7 +17,7 @@ import { mkdir, readdir, stat, unlink, writeFile } from 'node:fs/promises'
 import { VIEW, DESKTOP_UA } from './sidecar.js?v=1'
 import { planOf, gather, sizeText, coreRows } from './media.js?m=2'
 import { dropLogin, listLogins, readLogin, saveLogin, vaultState } from './vault.js'
-import { readPage, renderPage, readArticle, readRelevant, findOnPage, locate, locateFirst, FRAME_BASE, covering, contextFor, identityOf, sameThing, differs, labelSpot, selectSpot, bounds, pictures, clickInDom, writeInDom, REF_GONE } from './read.js?v=1'
+import { readPage, renderPage, readArticle, readRelevant, readTabs, findOnPage, locate, locateFirst, FRAME_BASE, covering, contextFor, identityOf, sameThing, differs, labelSpot, selectSpot, bounds, pictures, clickInDom, writeInDom, REF_GONE } from './read.js?v=1'
 import { brief as askBrief, recordRound, lastRoundNote, lastRound, markRound, MARK_STATUS } from './review.js?v=1'
 
 /**
@@ -2233,6 +2233,7 @@ export function sidecarTools(sidecar, refs, host = {}) { // ui-copy-ignore
           max_chars: { type: 'integer', description: 'Character ceiling. Default 6000; the result says when it was cut.' },
           query: { type: 'string', description: 'What you want to know. With a query, only the passages that answer it come back, ranked — the cheapest way to read a long page. Without one, the whole page comes back.' },
           force: { type: 'boolean', description: 'Read again even if the page has not changed since this chat last read it.' },
+          across: { type: 'boolean', description: 'When this tab has no answer, also read the other tabs this chat holds and answer from whichever has it. Default true; pass false to stay on this one.' },
         },
         additionalProperties: false,
       },
@@ -2260,13 +2261,43 @@ export function sidecarTools(sidecar, refs, host = {}) { // ui-copy-ignore
           const found = await readRelevant(page, asked, { maxChars: ceiling })
           if (found.fault !== undefined) return `the page did not answer when read — ${String(found.fault).split('\n')[0].slice(0, 200)}`
           if (found.text === '') {
-            /* The question missed every block: the page is still the answer, so
-               hand it over whole rather than make the caller guess again. */
+            /* The answer may be in the tab behind this one. Reading every tab
+               this chat holds is one call, where switching and reading again is
+               two, so the miss tries the others before it gives up. */
+            let note = ''
+            if (args?.across !== false) {
+              const others = await sidecar.elsewhere(currentSession()).catch(() => [])
+              if (others.length > 0) {
+                try {
+                  const pooled = await readTabs(others, asked, { maxChars: ceiling })
+                  if (pooled.rows.length > 0) {
+                    const body = pooled.rows
+                      .map((row) => `## ${String(row.tab)}\n${String(row.text)}`)
+                      .join('\n\n')
+                    const from = pooled.tabs.length === 1 ? 'another tab' : `${String(pooled.tabs.length)} other tabs`
+                    return `${found.url}\n# ${found.title === '' ? '(untitled)' : found.title}\n\nthis tab does not answer ${JSON.stringify(asked)}; the answer is in ${from} this chat holds (${String(pooled.matched)} passages):\n\n${body}`
+                  }
+                  note = ` (${String(pooled.tabs.length)} other tab${pooled.tabs.length === 1 ? '' : 's'} read too, none answers it)`
+                } finally {
+                  for (const one of others) one.close()
+                }
+              }
+            }
+            /* A question the page does not answer costs the nearest passages and
+               a sentence saying so, not the whole page: whole pages are what
+               this tool exists to avoid, and the caller can ask for one. */
+            const closest = Array.isArray(found.nearby) ? found.nearby : []
+            if (closest.length > 0) {
+              const body = closest
+                .map((row) => `${row.heading === '' ? '' : `## ${String(row.heading)}\n`}${String(row.text)}`)
+                .join('\n\n')
+              return keep(`${found.url}\n# ${found.title === '' ? '(untitled)' : found.title}\n\nnothing directly answers ${JSON.stringify(asked)} (${String(found.total)} blocks read).${note} The closest passages:\n\n${body}\n\nask for the whole page with browser_view, or try other words.`)
+            }
             const whole = await readArticle(page, { maxChars: ceiling })
             if (whole.fault === undefined && whole.text !== '') {
-              return keep(`${whole.url}\n# ${whole.title === '' ? '(untitled)' : whole.title}\n\nnothing here answers ${JSON.stringify(asked)} (${String(found.total)} blocks read), so the whole page follows:\n\n${whole.text}${whole.truncated ? `\n\n(cut at ${String(whole.chars)} characters; raise max_chars for more)` : ''}`)
+              return keep(`${whole.url}\n# ${whole.title === '' ? '(untitled)' : whole.title}\n\nnothing here answers ${JSON.stringify(asked)} (${String(found.total)} blocks read).${note} so the whole page follows:\n\n${whole.text}${whole.truncated ? `\n\n(cut at ${String(whole.chars)} characters; raise max_chars for more)` : ''}`)
             }
-            return `${found.url}\n# ${found.title === '' ? '(untitled)' : found.title}\n\nnothing here answers ${JSON.stringify(asked)} (${String(found.total)} blocks read). Call browser_view for the whole page, or try other words.`
+            return `${found.url}\n# ${found.title === '' ? '(untitled)' : found.title}\n\nnothing here answers ${JSON.stringify(asked)} (${String(found.total)} blocks read).${note} Call browser_view for the whole page, or try other words.`
           }
           return keep(`${found.url}\n# ${found.title === '' ? '(untitled)' : found.title}\n\n${found.text}\n\n(${String(found.matched)} of ${String(found.total)} blocks, ranked for ${JSON.stringify(asked)})`)
         }