symbiot 0.41.7 → 0.42.0

package/README.md

@@ -60,9 +60,13 @@ Tabs:
   how many new since yesterday (or in the last 3 or 7 days), its latest brief, the
   newest items, and **Open**, **Draft a reply** and **Check now** on them. **Seen**
   on a card sets its count back to 0 (what it found stays under Watching, and the
-  other cards keep theirs). The tab shows the count, so you see what's waiting from
+  other cards keep theirs). **💬** on a card opens a chat with your AI about what's
+  new there: what needs you, and what to say to whom. Agree it there, then **Draft
+  a reply** on one, and your agent gets that talk with the message, so it writes
+  what you agreed. The tab shows the count, so you see what's waiting from
   any tab, and once you watch something the app opens on the Dashboard instead of
-  the Map. `symbiot watch board` prints the same cards as JSON.
+  the Map. `symbiot watch board` prints the same cards as JSON, and
+  `symbiot watch board --line` as one line for a status bar (`2 emails · 1 WhatsApp message`).
 - **Drift** — the [`symbiot drift`](#whats-out-of-sync-symbiot-drift) report, with
   a "fetch latest" toggle (and an [experimental](#experimental) "check CI").
 - **Week / Standup / Todo** — the write-ups (these use your chosen AI).
@@ -81,7 +85,7 @@ Tabs:
   did** — files changed and commits made, read from git, whichever agent it was.
   When an agent leaves **questions, options or ideas** for you, they show up on its
   block (see [Questions from your agent](#questions-from-your-agent)).
-- **Settings** — your AI, the folders to scan, your agent command, the
+- **Settings** — [Link your work](#link-your-work), your AI, the folders to scan, your agent command, the
   [weekly write-up and start at login](#every-week-and-at-login), plus the
   [experimental](#experimental) email, model recommendations and one-click local
   model.
@@ -389,7 +393,16 @@ that starts with `[repo: <name>]`, to that repo's Tasks: an agent working on one
 project can have ideas for another (Symbiot itself, say). The agent is told to
 keep going with everything that doesn't depend on an answer, and to ask instead of
 doing anything destructive. Both files live in `.symbiot/`, so they're never
-committed.
+committed. The questions stay on the Agents tab after Symbiot restarts (or when
+`symbiot push --open` started the run): Symbiot notes each folder a run starts in
+(`~/.config/symbiot/runs.json`) and lists the ones still waiting on you.
+
+**A 👤 You answer is your step.** Picking an option that starts with **You** saves
+the answer and reminds you the step is still yours to do. Your agent waits for it
+instead of starting straight away and asking the same thing again. If the step
+names a file in backticks (`` `.env` ``), the agent starts by itself once that
+file changes, while the app runs. Otherwise **Start it now** on that folder's
+block starts it once you've done the step.
 
 **A run that stopped on questions isn't repeated for nothing.** When a run ends
 having asked questions you haven't answered, with tasks still unticked, Symbiot
@@ -426,6 +439,40 @@ repo), keyed by repo path or folder name, that prints the deployed commit sha:
 `{ "/path/to/repo": "ssh prod cat ~/app/.deployed-sha" }`. Also the **Drift** tab
 in the app.
 
+## Link your work
+
+Settings → **Link your work** has one button per standard work site: Gmail,
+Outlook, Google and Outlook Calendar, GitHub, GitLab, Slack, Microsoft Teams,
+WhatsApp, Jira & Confluence, Linear, Asana, Trello, Google Drive, Notion,
+HubSpot and Salesforce. One click:
+
+1. opens the site in Symbiot's own browser, where you sign in the usual way (its
+   own login, SSO and 2FA; Symbiot never sees your password). Close the window
+   when you're signed in;
+2. trusts the site, so [Screens](#screens-blueprints-for-screen-automation)' Press
+   and Type work there without asking;
+3. watches its inbox or notifications, so what arrives shows on the **Dashboard**,
+   in Standup's "Waiting on you", and in **Week**. Someone who doesn't write code
+   gets a Week from their linked sites too.
+
+Each button shows where it stands: grey (not linked), amber (sign in, or signed
+out), green (linked), with ↻ to check now and × to unlink. Unlinking stops
+watching it and stops trusting what the link trusted, never a site you trusted
+yourself.
+
+**For a whole company**, one `links.json` in Symbiot's config folder (or the file
+`SYMBIOT_LINKS` names) gives everyone the same buttons. It adds your own sites and
+hides the ones you don't use. An entry with a built-in's id replaces it:
+
+```json
+{ "links": [{ "id": "jira", "name": "Jira", "url": "https://acme.atlassian.net/jira/your-work", "watch": true },
+            { "name": "Acme CRM", "url": "https://crm.acme.example/inbox", "hosts": ["sso.acme.example"], "watch": true }],
+  "hide": ["whatsapp", "trello"] }
+```
+
+Linking stays in the app. Like trusted sites, there's no command an agent could
+call to link a site itself.
+
 ## Connect it (once)
 
 Pick the AI you want it to write with:
@@ -627,11 +674,14 @@ browser and maps what's in the window then, as a new screen. It scrolls what you
 mouse wheel would: the page, or the part of it that scrolls on its own (Gmail's
 list of mail). Scrolling only looks, so it never asks first. Or **Whole page**
 maps all of it at once: one tall screenshot (up to 16,000 pixels) with every
-button, link and field on it marked where it is on the page. It's for a page that
-scrolls as a whole; a list that scrolls inside the page, like Gmail's, still takes
-Scroll down. The hidden browser scrolls down a window at a time and puts the
-screenshots together, so a menu bar that stays at the top shows once, at the top,
-and a sticky side menu shows in the first window only. **Press** on a region clicks it
+button, link and field on it marked where it is on the page. The hidden browser
+scrolls down a window at a time and puts the screenshots together, so a menu bar
+that stays at the top shows once, at the top, and a sticky side menu shows in the
+first window only. Where a list scrolls inside the page instead, like Gmail's
+mail, Whole page opens that list out: it scrolls the list a part at a time and
+puts the parts together, with the rest of the window (the search bar, Compose,
+your folders) around it once, so a whole inbox page is one screen with every row
+marked. **Press** on a region clicks it
 in that hidden browser and maps the page it leads to as a new screen, so map,
 press, map is how an agent finds its way around a site. **Type** on a field types
 your text into it there (replacing what was in it) and, with **Type, then Enter**
@@ -695,6 +745,7 @@ symbiot watch add <screen id> --every 15   # watch a mapped page (or: symbiot wa
 symbiot watch add github                   # your GitHub notifications (see below)
 symbiot watch new --hours 24               # what's new, newest first (JSON)
 symbiot watch board                        # the Dashboard's cards (JSON; .total is the count, for a status bar)
+symbiot watch board --line                 # the same as one line: "2 emails · 1 WhatsApp message" (empty when nothing's new)
 symbiot watch seen <watch id>              # set a card back to 0, like its Seen button
 symbiot watch check                        # read them all now (JSON)
 symbiot watch draft <id>                   # Draft a reply to a new email or chat message (see below)
@@ -737,8 +788,9 @@ emails `"mail": true`, and `symbiot watch draft <id>` does the same as the butto
 A new WhatsApp message (a watched `web.whatsapp.com`) has **Draft a reply** too.
 There, the agent opens the chat in the hidden browser and types the reply into
 its message box, and leaves it there unsent. WhatsApp keeps it as that chat's
-draft in Symbiot's browser: type `web.whatsapp.com` under Screens and click
-**Sign in** to read it and send it. The same guard holds, plus one for chats: a
+draft in Symbiot's browser: once the agent is done, **Open in WhatsApp** on the
+message opens that browser at `web.whatsapp.com` as a window, to read the reply
+and send it (close the window after, so Watch can read WhatsApp again). The same guard holds, plus one for chats: a
 draft's run can't press Enter (in a chat, Enter sends), and in WhatsApp its line
 breaks are typed as spaces. Add `web.whatsapp.com` under Trusted sites for it.
 `symbiot watch new` marks those `"chat": true`.

package/agents.mjs

@@ -3,10 +3,10 @@
 // tab, and the questions an unattended agent leaves for the user.
 import { spawn } from "node:child_process";
 import { homedir } from "node:os";
-import { join } from "node:path";
+import { join, dirname } from "node:path";
 import { readFileSync, writeFileSync, mkdirSync, existsSync, openSync, writeSync, unlinkSync, readdirSync, statSync } from "node:fs";
 import { randomBytes, createHash } from "node:crypto";
-import { loadConfig, saveConfig, loadTasks, sameTask, taskWords, sh, hasCmd } from "./core.mjs";
+import { CONFIG_DIR, loadConfig, saveConfig, loadTasks, sameTask, taskWords, sh, hasCmd } from "./core.mjs";
 
 const HANDOFFS = []; // live registry of agents Symbiot has handed work to
 // ---- hand a repo (+ its tasks) to the user's agent — generic, settings-based
@@ -112,6 +112,8 @@ function runHandoff(repoPath, { force = false } = {}) {
   const busy = runningHandoff(repoPath); if (busy) return { busy: true, id: busy.id || "", pid: busy.pid, auto: !!busy.auto };
   releaseHeldTasks(repoPath); // held for an agent another process started, which has since exited
   tmpl = withConnectors(tmpl, repoPath);
+  if (force) { try { unlinkSync(join(repoPath, ".symbiot", WAITING)); } catch {} }
+  else { const w = waitingFor(repoPath); if (w) return { blocked: true, waiting: true, questions: 0, note: waitNote(w) }; }
   const blocked = !force && blockedAgain(repoPath, tmpl); if (blocked) return blocked;
   const lock = join(repoPath, ".symbiot", LOCK);
   const env = opts.env && typeof opts.env === "object" ? Object.fromEntries(Object.entries(opts.env).map(([k, v]) => [k, String(v)])) : null;
@@ -123,8 +125,92 @@ function runHandoff(repoPath, { force = false } = {}) {
   if (!e) return null;
   e.handoff = true;
   if (e.pid) try { writeFileSync(lock, JSON.stringify({ pid: e.pid, id: e.id, startedAt: e.startedAt, owner: process.pid })); } catch {}
+  noteRun(e);
   return e;
 }
+// ---- runs from before the app started ---------------------------------------
+// HANDOFFS lives in memory, so after a restart a run that stopped on questions
+// was gone from the Agents tab, its questions with it: there was no answering
+// them there, only Start it anyway or a change to .env. So each folder a run
+// starts in is noted in ~/.config/symbiot/runs.json ({ path, name, startedAt },
+// newest first, by whichever process started it: the app, or `symbiot push
+// --open`), and the Agents tab lists the ones it isn't tracking itself while they
+// have questions open, a step of yours to wait for, or a run still going.
+const RUNS_FILE = join(CONFIG_DIR, "runs.json"), RUNS_MAX = 60;
+function loadRuns() { try { const a = JSON.parse(readFileSync(RUNS_FILE, "utf8")); return Array.isArray(a) ? a.filter((r) => r && typeof r.path === "string" && r.path) : []; } catch { return []; } }
+function noteRun(e) {
+  const list = [{ path: e.path, name: e.name, startedAt: e.startedAt }, ...loadRuns().filter((r) => r.path !== e.path)].slice(0, RUNS_MAX);
+  try { mkdirSync(CONFIG_DIR, { recursive: true }); writeFileSync(RUNS_FILE, JSON.stringify(list, null, 2), { mode: 0o600 }); } catch {}
+}
+const knownRun = (path) => HANDOFFS.some((e) => e.path === path) || loadRuns().some((r) => r.path === path);
+// The Agents tab's block for a run another process (or this app before a
+// restart) started: as agentsList gives one, marked `earlier`. Its end is when
+// its log last changed; its exit code isn't known.
+function earlierRuns() {
+  const tracked = new Set(HANDOFFS.map((e) => e.path)), out = [];
+  for (const r of loadRuns()) {
+    if (tracked.has(r.path) || !existsSync(join(r.path, ".symbiot"))) continue;
+    const busy = runningHandoff(r.path), ask = agentQuestions(r.path, r.name), wait = waitingFor(r.path);
+    if (!busy && !ask.questions.length && !wait) continue;
+    const log = join(r.path, ".symbiot", "agent.log"), startedAt = (busy && busy.startedAt) || Number(r.startedAt) || Date.now();
+    let tail = "", end = Date.now(); try { tail = readFileSync(log, "utf8").slice(-1200); if (!busy) end = statSync(log).mtimeMs; } catch {}
+    out.push({ id: "earlier-" + digest(r.path).slice(0, 8), name: String(r.name || r.path.split("/").pop()), path: r.path, status: busy ? "running" : "done", earlier: true, elapsed: Math.max(0, end - startedAt), exitCode: null, tail, changed: agentChanges(r.path, startedAt), ask, held: heldTasks(r.path), waiting: wait, fromHeld: false });
+  }
+  return out;
+}
+
+// ---- a step that's yours: wait for it -----------------------------------------
+// An answer that picks a "👤 You:" option leaves a step for the user to do (put a
+// key in .env, allow a command in Settings). Starting the agent before it's done
+// only gets the same questions back. So answerQuestions reminds them the step is
+// theirs, and Send answers & continue waits instead of starting: .symbiot/
+// waiting.json keeps the step and how each file it names (in backticks, like
+// `.env`) was then. Once one of them changes, the next run goes ahead
+// (startWaiting starts the one that was asked for); with no file to watch, it
+// waits for Start it now. Start it anyway (force) always goes ahead.
+const WAITING = "waiting.json";
+const yourStep = (a) => /👤/.test(String(a || ""));
+// The files a step names, in backticks: `.env`, `~/.termux/termux.properties`,
+// `.claude/settings.json`. Not a command (`npm test`), a site (`web.whatsapp.com`,
+// unless there's a file by that name) or a path that can't be one here.
+function namedFiles(text, dir) {
+  const out = [];
+  for (const m of String(text || "").matchAll(/`([^`\s]+)`/g)) {
+    const t = m[1].replace(/[.,:;]+$/, ""), first = t.replace(/^~?\//, "").split("/")[0];
+    if (/:\/\//.test(t) || !/[./]/.test(t) || /^[@\d]/.test(t) || /^\.{1,2}\/?$/.test(t)) continue;
+    const p = t.startsWith("~/") ? join(homedir(), t.slice(2)) : t.startsWith("/") ? t : join(dir, t);
+    const dotted = first.startsWith(".") && first !== "." && first !== "..";
+    const host = !dotted && /\./.test(first) && !t.startsWith("~/") && !t.startsWith("/");
+    if (existsSync(p) || (!host && (dotted || existsSync(dirname(p))))) out.push({ name: t, path: p });
+  }
+  return out.filter((f, i) => out.findIndex((g) => g.path === f.path) === i).slice(0, 5);
+}
+const fileSig = (p) => { try { const s = statSync(p); return s.mtimeMs + ":" + s.size; } catch { return "none"; } };
+// What a folder is waiting on: { step, files: [names], at, rerun }, or null (and
+// waiting.json goes) once a file it names has changed.
+function waitingFor(path) {
+  let w = null; try { w = JSON.parse(readSymbiot(path, WAITING)); } catch {}
+  if (!w || typeof w.step !== "string") return null;
+  const files = Array.isArray(w.files) ? w.files : [];
+  if (files.some((f) => f && fileSig(f.path) !== f.sig)) { try { unlinkSync(join(path, ".symbiot", WAITING)); } catch {} return null; }
+  return { step: w.step, files: files.map((f) => f.name), at: w.at || 0, rerun: !!w.rerun };
+}
+function waitNote(w) {
+  const names = w.files.map((f) => "`" + f + "`").join(" or ");
+  return `Not started: your step comes first: ${w.step.replace(/^\s*👤\s*(You:)?\s*/, "")} ${names ? `Your agent starts once ${names} changes.` : "Once it's done, click Start it now (or Start it anyway)."}`;
+}
+// Runs that waited for a file of yours, and were asked for, start once it changes.
+// The app calls this every so often. Returns the jobs started.
+function startWaiting() {
+  const started = [];
+  for (const r of loadRuns()) {
+    let w = null; try { w = JSON.parse(readSymbiot(r.path, WAITING)); } catch {}
+    if (!w || !w.rerun || !Array.isArray(w.files) || !w.files.length || runningHandoff(r.path)) continue;
+    if (waitingFor(r.path)) continue; // nothing it names has changed yet
+    const e = runHandoff(r.path); if (e && !e.busy && !e.blocked) started.push(e);
+  }
+  return started;
+}
 // ---- a run that would only report the same blockers again -------------------
 // A run that ends having asked questions (QUESTIONS.md written during it, some
 // still unanswered) with tasks left unticked is blocked on the user: noteBlocked
@@ -400,9 +486,13 @@ function agentQuestions(path, repo) {
 }
 // Save answers to .symbiot/ANSWERS.md; opts.rerun hands the repo back to the
 // agent (same saved command) so it carries on with them.
+// An answer that picks a "👤 You:" option is a step for the user: they're told
+// it's still theirs (`yours`), and the next run waits for it (waiting.json).
+// A folder a run started in before a restart can be answered too (runs.json).
+const stepText = (a) => String(a).split("🤖")[0].replace(/^\s*👤\s*(You:)?\s*/, "").replace(/\s*\(recommended\)\s*$/i, "").trim();
 function answerQuestions(path, answers, opts = {}) {
   path = String(path || "");
-  if (!path || !HANDOFFS.some((e) => e.path === path)) return { error: "No agent has run in that folder." };
+  if (!path || !knownRun(path)) return { error: "No agent has run in that folder." };
   const open = new Map(agentQuestions(path, "").questions.map((x) => [qKey(x.q), x.q]));
   const rows = (Array.isArray(answers) ? answers : [])
     .map((x) => ({ q: open.get(qKey(x && x.q)), a: String((x && x.a) || "").replace(/^\s*#+/gm, "").trim().slice(0, 2000) }))
@@ -415,6 +505,15 @@ function answerQuestions(path, answers, opts = {}) {
     writeFileSync(join(path, ".symbiot", "ANSWERS.md"), prev.replace(/\s*$/, "\n") + rows.map((x) => `\n### ${x.q}\n${x.a}\n_answered ${day}_\n`).join(""));
   } catch (e) { return { error: "Couldn't write ANSWERS.md: " + ((e && e.message) || e) }; }
   const out = { ok: true, saved: rows.length };
+  const yours = rows.filter((x) => yourStep(x.a)).map((x) => stepText(x.a)).filter(Boolean);
+  if (yours.length) {
+    const step = yours.join(" "), files = namedFiles(rows.filter((x) => yourStep(x.a)).map((x) => x.a.split("🤖")[0]).join(" "), path);
+    try { writeFileSync(join(path, ".symbiot", WAITING), JSON.stringify({ step, files: files.map((f) => ({ ...f, sig: fileSig(f.path) })), at: Date.now(), rerun: !!opts.rerun })); } catch {}
+    const names = files.map((f) => "`" + f.name + "`").join(" or ");
+    out.yours = yours; if (files.length) out.waitFiles = files.map((f) => f.name);
+    out.note = "Answers saved. That step is still yours to do. " + (files.length ? `Your agent ${opts.rerun ? "starts by itself" : "can start"} once ${names} changes${opts.rerun ? " (while Symbiot runs)" : ""}.` : "Once it's done, click Start it now on this folder.");
+    return out;
+  }
   if (opts.rerun) { const e = runHandoff(path); if (e && e.busy) out.note = "Answers saved. An agent is still running in that folder, so another wasn't started. Send them again once it finishes."; else if (e && e.blocked) out.note = "Answers saved. " + e.note; else if (e) out.rerun = e.id; else out.note = "Answers saved. Set an agent command in Settings to have the agent pick them up automatically."; }
   return out;
 }
@@ -424,14 +523,16 @@ function heldTasks(path) {
   return md ? md.split("\n").filter((l) => /^\s*-\s*\[ \]/.test(l)).map((l) => l.replace(/^\s*-\s*\[ \]\s*/, "").trim().slice(0, 200)) : null;
 }
 // The Agents tab: every tracked job, its log tail, what it changed, and — on the
-// newest job per folder — the questions/ideas it left and any tasks held for it.
+// newest job per folder — the questions/ideas it left, any tasks held for it and
+// a step of yours it waits on. Then runs from before the app started that still
+// need you (earlierRuns).
 function agentsList() {
   const seen = new Set();
   return HANDOFFS.map((e) => {
     let tail = ""; try { tail = readFileSync(e.log, "utf8").slice(-1200); } catch {}
     const first = !seen.has(e.path); seen.add(e.path);
-    return { id: e.id, name: e.name, path: e.path, status: e.status, elapsed: (e.endedAt || Date.now()) - e.startedAt, exitCode: e.exitCode, tail, changed: agentChanges(e.path, e.startedAt), ask: first ? agentQuestions(e.path, e.name) : null, held: first ? heldTasks(e.path) : null, fromHeld: !!e.fromHeld };
-  });
+    return { id: e.id, name: e.name, path: e.path, status: e.status, elapsed: (e.endedAt || Date.now()) - e.startedAt, exitCode: e.exitCode, tail, changed: agentChanges(e.path, e.startedAt), ask: first ? agentQuestions(e.path, e.name) : null, held: first ? heldTasks(e.path) : null, waiting: first && e.status !== "running" ? waitingFor(e.path) : null, fromHeld: !!e.fromHeld };
+  }).concat(earlierRuns());
 }
 
-export { HANDOFFS, HANDOFF_PROMPT, QUESTIONS_MAX, shSingle, CLAUDE_CMD, ORCA_CLAUDE_CMD, handoffCmd, setHandoffCmd, grantAgent, grantRule, allowTool, claudeConnectors, withConnectors, connectorsInfo, fillHandoff, runHandoff, blockedAgain, runningHandoff, writeTasks, releaseHeldTasks, startHeldTasks, detectHandoffs, orcaHandoffCmd, migrateOrcaCmd, migrateClaudeCmd, track, agentChanges, parseQuestions, suggestionTarget, agentQuestions, answerQuestions, agentsList };
+export { HANDOFFS, HANDOFF_PROMPT, QUESTIONS_MAX, shSingle, CLAUDE_CMD, ORCA_CLAUDE_CMD, handoffCmd, setHandoffCmd, grantAgent, grantRule, allowTool, claudeConnectors, withConnectors, connectorsInfo, fillHandoff, runHandoff, blockedAgain, runningHandoff, loadRuns, earlierRuns, namedFiles, waitingFor, startWaiting, writeTasks, releaseHeldTasks, startHeldTasks, detectHandoffs, orcaHandoffCmd, migrateOrcaCmd, migrateClaudeCmd, track, agentChanges, parseQuestions, suggestionTarget, agentQuestions, answerQuestions, agentsList };

package/headless.mjs

@@ -8,7 +8,8 @@
 // app the browser stays open for a few minutes after each action, so the next
 // one carries on from the page as it is (type into a field, then press Send).
 // A map has what fits in the window: Scroll down maps the next part, and Whole
-// page maps all of a page that scrolls as a whole in one tall screenshot.
+// page maps all of it in one tall screenshot: the page, or, where a list scrolls
+// inside it (Gmail's mail), that list opened out to its full length.
 // readPage reads a page without saving a screen, for Watch (watch.mjs).
 //
 // It drives Chrome / Chromium / Edge / Brave over the DevTools protocol on a pipe
@@ -21,7 +22,7 @@ import { spawn } from "node:child_process";
 import { join } from "node:path";
 import { mkdirSync, existsSync } from "node:fs";
 import { CONFIG_DIR, chromeBinary, loadConfig, saveConfig } from "./core.mjs";
-import { loadScreens, addPageScreen, center, stitchPng } from "./screens.mjs";
+import { loadScreens, addPageScreen, center, stitchPng, stitchListPng } from "./screens.mjs";
 
 const PROFILE = join(CONFIG_DIR, "browser");
 const VIEW = { w: 1280, h: 800 }; // the page's size: a laptop-sized window
@@ -212,11 +213,19 @@ const SCROLLER = `function scroller() {
 // whole: called again a window further down each time (snapshotWhole), it adds
 // what's new on screen to what it found before (kept in the page) and gives all
 // of it, each where it is on the page, not the window.
-const collectJs = (whole) => `(() => { ${SCROLLER}
+// list: { selector, part, more } for a list that scrolls inside the page
+// (snapshotList). part "list": what's in the list, called again as it scrolls
+// and kept like whole's, each where it is with the list opened out (its y plus
+// how far the list is scrolled), cut to the list's box. part "rest": the rest of
+// the window, what's below the list moved down by `more`.
+const collectJs = (whole, list = null) => `(() => { ${SCROLLER}
   const SEL = 'a[href],button,input:not([type=hidden]),select,textarea,summary,[contenteditable=""],[contenteditable=true],[onclick],' +
     ['button','link','tab','menuitem','menuitemcheckbox','menuitemradio','option','checkbox','radio','switch','combobox','textbox','searchbox','row','treeitem'].map((r) => '[role=' + r + ']').join(',');
   const W = innerWidth, H = innerHeight, X = ${whole ? "scrollX" : "0"}, Y = ${whole ? "scrollY" : "0"};
-  const kept = ${whole ? "(window.__symbiotWhole = window.__symbiotWhole || { out: [], kinds: new Map() })" : "{ out: [], kinds: new Map() }"}, out = kept.out, kinds = kept.kinds;
+  const IN = ${JSON.stringify(list)}, LIST = IN ? document.querySelector(IN.selector) : null;
+  const LB = LIST && (() => { const b = LIST.getBoundingClientRect(); return { left: Math.max(0, b.left), top: Math.max(0, b.top), right: Math.min(W, b.right), bottom: Math.min(H, b.bottom) }; })();
+  if (IN && !LIST) return { url: location.href, title: document.title, items: [] };
+  const kept = ${whole || (list && list.part === "list") ? "(window.__symbiotWhole = window.__symbiotWhole || { out: [], kinds: new Map(), at: new Set() })" : "{ out: [], kinds: new Map(), at: new Set() }"}, out = kept.out, kinds = kept.kinds;
   const clean = (s) => String(s || '').replace(/\\s+/g, ' ').trim();
   const kindOf = (e) => {
     const t = e.tagName.toLowerCase(), r = (e.getAttribute('role') || '').toLowerCase(), ty = (e.getAttribute('type') || 'text').toLowerCase();
@@ -257,11 +266,18 @@ const collectJs = (whole) => `(() => { ${SCROLLER}
   };
   for (const e of document.querySelectorAll(SEL)) {
     if (out.length >= 200) break;
-    if (kinds.has(e) || e.disabled || e.closest('[aria-hidden=true],[inert]')) continue;
+    const inList = !!LIST && LIST.contains(e);
+    if (IN && inList !== (IN.part === 'list')) continue;
+    // in an opened-out list, a row already marked is known by where it is, not by its
+    // element: a list that reuses its rows as it scrolls shows other mail in the same one
+    if ((!inList && kinds.has(e)) || e.disabled || e.closest('[aria-hidden=true],[inert]')) continue;
     const kind = kindOf(e), up = e.parentElement && e.parentElement.closest(SEL);
     if (up && kinds.get(up) === kind && kind !== 'field') continue;
-    const b = e.getBoundingClientRect();
-    const x = Math.max(0, b.left), y = Math.max(0, b.top), x2 = Math.min(W, b.right), y2 = Math.min(H, b.bottom);
+    const b = e.getBoundingClientRect(), CL = inList ? LB : { left: 0, top: 0, right: W, bottom: H };
+    // cut off at the list's bottom: the next part shows all of it (it overlaps), unless this is the
+    // last; cut off at its top, the part before showed all of it
+    if (inList && b.height <= LB.bottom - LB.top && ((!IN.last && b.bottom > LB.bottom + 1) || (LIST.scrollTop > 0 && b.top < LB.top - 1))) continue;
+    const x = Math.max(CL.left, b.left), y = Math.max(CL.top, b.top), x2 = Math.min(CL.right, b.right), y2 = Math.min(CL.bottom, b.bottom);
     if (x2 - x < 4 || y2 - y < 4) continue;
     const st = getComputedStyle(e);
     if (st.visibility === 'hidden' || st.pointerEvents === 'none' || +st.opacity === 0) continue;
@@ -269,10 +285,14 @@ const collectJs = (whole) => `(() => { ${SCROLLER}
     const top = document.elementFromPoint((x + x2) / 2, (y + y2) / 2);
     if (!top || !(top === e || e.contains(top) || top.contains(e))) continue;
     ${whole ? "if (y >= H - 200 && stuck(e)) continue; // stuck to the window's bottom: left out of the whole page's picture" : ""}
-    kinds.set(e, kind);
-    const full = labelOf(e);
-    const box = ${whole ? "{ x: Math.max(0, b.left + X), y: Math.max(0, b.top + Y), w: b.width, h: b.height }" : "{ x, y, w: x2 - x, h: y2 - y }"};
-    const r = { label: full.slice(0, 80) || kind, kind, x: Math.round(box.x), y: Math.round(box.y), w: Math.round(box.w), h: Math.round(box.h), selector: cssPath(e) };
+    const full = labelOf(e), at = inList ? Math.round(b.top + LIST.scrollTop) + '|' + kind + '|' + full : '';
+    if (at && kept.at.has(at)) { kinds.set(e, kind); continue; }
+    // an element marked before, showing something else now: its selector would find it, not this
+    const reused = inList && kinds.has(e);
+    kinds.set(e, kind); if (at) kept.at.add(at);
+    const down = !IN ? 0 : inList ? LIST.scrollTop : y >= LB.bottom ? IN.more || 0 : 0;
+    const box = ${whole ? "{ x: Math.max(0, b.left + X), y: Math.max(0, b.top + Y), w: b.width, h: b.height }" : "{ x, y: y + down, w: x2 - x, h: y2 - y }"};
+    const r = { label: full.slice(0, 80) || kind, kind, x: Math.round(box.x), y: Math.round(box.y), w: Math.round(box.w), h: Math.round(box.h), selector: reused ? '' : cssPath(e) };
     if (full.length > 80) r.text = full.slice(0, 400); // all of a long one (an inbox row), for Watch
     if (e.href && /^https?:/.test(e.href)) r.href = String(e.href).slice(0, 500);
     out.push(r);
@@ -318,9 +338,9 @@ async function snapshot(page, name) {
 // What's fixed or sticky (a menu bar, a cookie banner) is hidden once it's in one,
 // so it shows once, where it was, not on every part. The parts overlap by
 // OVERLAP pixels, and the later one's are kept, so what stuck to the bottom of the
-// window is left out (and not marked: collectJs). Only for a page that scrolls
-// as a whole: a list that scrolls inside the page (Gmail's) wouldn't show more, so
-// that gives { error, inside } and is left to Scroll down.
+// window is left out (and not marked: collectJs). Where a list scrolls inside the
+// page instead (Gmail's), scrolling the page wouldn't show more: snapshotList
+// opens that list out (or gives { error, inside } when it can't find it).
 const MAX_TALL = 16000, OVERLAP = 200; // pixels
 const PAGE_HEIGHT = `(() => { ${SCROLLER}
   const d = document.scrollingElement || document.documentElement, sc = scroller();
@@ -341,7 +361,7 @@ async function evaluate(page, expression) {
 }
 async function snapshotWhole(page, name) {
   const first = await evaluate(page, PAGE_HEIGHT);
-  if (first.inside) return { error: "This page doesn't scroll as a whole: a list inside it does (like Gmail's mail), and a tall screenshot wouldn't show more of that. Use Scroll down to map the next part of the list.", inside: true };
+  if (first.inside) return snapshotList(page, name);
   if (first.h <= VIEW.h + 1) { const s = await snapshot(page, name); return s.error || s.note ? s : { ...s, note: "It all fits in the window, so this map is the whole page." }; }
   // a window at a time, overlapping a little; the page can load more as it goes (up to MAX_TALL)
   let info = null, h = first.h, end = 0; const shots = [];
@@ -367,6 +387,44 @@ async function snapshotWhole(page, name) {
   const note = [h > tall + 1 && `The page goes on past ${tall} pixels, so this map stops there.`, items.length >= 200 && "It marks the first 200 buttons, links and fields, from the top."].filter(Boolean).join(" ");
   return note ? { ...s, note } : s;
 }
+// The whole of a page whose list scrolls inside it (Gmail's mail), in one tall
+// screenshot: the list opened out to its full length (up to MAX_TALL), the rest
+// of the window as it is, and what's below the list moved down. The list is
+// scrolled a box at a time, overlapping a little, and each part put in its place
+// (screens.mjs stitchListPng). Every row in it is a region, where it is in the
+// tall picture; what's around the list (Compose, the folders) is marked first.
+// The page itself doesn't move, so the window is as it was after.
+async function snapshotList(page, name) {
+  const top = await collect(page), sel = top.scroll && top.scroll.selector;
+  const box = sel && await evaluate(page, `(() => { const e = document.querySelector(${JSON.stringify(sel)}); if (!e) return null;
+    const b = e.getBoundingClientRect(), x = Math.max(0, Math.round(b.left)), y = Math.max(0, Math.round(b.top));
+    return { x, y, w: Math.min(innerWidth, Math.round(b.right)) - x, h: Math.min(innerHeight, Math.round(b.bottom)) - y }; })()`);
+  if (!box || box.w < 8 || box.h < 40) return { error: "Couldn't find the list that scrolls on this page. Use Scroll down to map the next part of it.", inside: true };
+  const room = MAX_TALL - VIEW.h, step = Math.max(20, box.h - Math.min(OVERLAP, Math.floor(box.h / 4)));
+  let list = null, rest = null, at = -1, max = 0; const shots = [];
+  await evaluate(page, "window.__symbiotWhole = null");
+  try {
+    for (let y = 0, i = 0; i < 100; i++) {
+      const m = await evaluate(page, scrollJs(y, sel)); if (!m || m.y <= at) break; // it wouldn't scroll further
+      await sleep(150);
+      at = m.y; max = m.max;
+      const last = at >= Math.min(max, room);
+      if (!list || list.items.length < 200) list = await collect(page, collectJs(false, { selector: sel, part: "list", last }));
+      const { data } = await page.send("Page.captureScreenshot", { format: "png" });
+      shots.push({ png: Buffer.from(data, "base64"), y: at });
+      if (last) break;
+      y = Math.min(at + step, Math.min(max, room));
+    }
+    rest = await collect(page, collectJs(false, { selector: sel, part: "rest", more: at }));
+  } finally { await evaluate(page, `window.__symbiotWhole = null; ${scrollJs(0, sel)}`).catch(() => {}); }
+  const png = shots.length && stitchListPng(shots, box, at);
+  if (!png) return { error: "Couldn't put the list's screenshots together." };
+  const tall = VIEW.h + at, items = [...rest.items, ...list.items].filter((r) => r.y < tall).slice(0, 200);
+  const s = saveScreen(name, png, { url: top.url, title: top.title, full: true, list: sel, items }, " (whole page)");
+  if (s.error || s.note) return s;
+  const note = [max > at + 1 && `The list goes on past ${tall} pixels, so this map stops there.`, items.length >= 200 && "It marks the first 200 buttons, links and fields: what's around the list, then the list from the top."].filter(Boolean).join(" ");
+  return note ? { ...s, note } : s;
+}
 
 // Map a site: open it in the hidden browser and save what's on it as a screen.
 // whole: all of the page in one tall screen (snapshotWhole); a page whose list
@@ -504,9 +562,14 @@ function actOnRegion(id, regionId, verb, confirmed, check, act) {
     const { result } = await page.send("Runtime.evaluate", { expression: find, returnByValue: true });
     const found = !!(result && result.value);
     // not found by its selector: the same spot. On a whole-page screen that spot is
-    // on the page, not the window: scroll it into the window first.
+    // on the page, not the window: scroll it into the window first (the list, on
+    // one whose list was opened out, if the spot is in it).
     let at = found ? result.value : center(r);
-    if (!found && s.page.full) { const y = await evaluate(page, `(scrollTo(0, ${at.y} - innerHeight / 2), scrollY)`); at = { x: at.x, y: at.y - (Number(y) || 0) }; }
+    if (!found && s.page.full) {
+      const y = await evaluate(page, s.page.list ? `(() => { const e = document.querySelector(${JSON.stringify(s.page.list)}); if (!e) return 0;
+        const b = e.getBoundingClientRect(); if (${at.y} < b.top) return 0; e.scrollTop = ${at.y} - b.top - e.clientHeight / 2; return e.scrollTop; })()` : `(scrollTo(0, ${at.y} - innerHeight / 2), scrollY)`);
+      at = { x: at.x, y: at.y - (Number(y) || 0) };
+    }
     const navigating = page.until("Page.frameStartedLoading", 1500);
     const extra = await act(page, at, r);
     if (await navigating) await page.until("Page.loadEventFired", 30000);

package/index.mjs

@@ -39,7 +39,7 @@ import { buildTasksMd, taskType, shipChanges, shipWithBump, bumpOffer, learnNpm,
 import { produce, mailState, setMail, sentMail } from "./writeups.mjs";
 import { loadScreens, screenImage, blueprint } from "./screens.mjs";
 import { mapPage, wholePage, pressRegion, typeRegion, scrollPage, signIn, isTrusted } from "./headless.mjs";
-import { watchState, addWatch, removeWatch, seenWatch, newsSince, markNews, checkWatch, setBrief, draftReply, watchBoard } from "./watch.mjs";
+import { watchState, addWatch, removeWatch, seenWatch, newsSince, markNews, checkWatch, setBrief, draftReply, watchBoard, boardLine } from "./watch.mjs";
 import { PORT as PHONE_PORT, phoneState, pairComputer, pollComputer, forgetComputer } from "./phone.mjs";
 import { startApp, updateCmd, isAppRunningWeekly } from "./server.mjs";
 
@@ -367,8 +367,8 @@ async function cmdScreens() {
                                                scroll the page, map what's in the window then
                                                (a map's "more" says there's more below or above)
   symbiot screens whole <id>                   map all of that page in one tall screenshot
-                                               (a page that scrolls as a whole, not a list
-                                               inside it like Gmail's: scroll that)
+                                               (where a list scrolls inside the page, like
+                                               Gmail's mail, that list opened out)
   symbiot screens signin <site>               sign in once, in Symbiot's browser window
   --yes is needed unless the page's site is under Trusted sites in the app's Settings.
   While the app runs, these use its hidden browser, which stays open a few minutes:
@@ -392,8 +392,9 @@ async function cmdWatch() {
     for (const n of news.slice(0, 20)) console.log(`  ${c.d(new Date(n.ts).toLocaleString())}  ${n.text.slice(0, 110)}  ${c.d(n.name.slice(0, 30))}`);
     return;
   }
-  // the Dashboard's cards, for an agent or a status bar (.total is the count on the Dashboard tab)
-  if (sub === "board") return out(watchBoard(Math.min(168, Math.max(1, hours))));
+  // the Dashboard's cards, for an agent or a status bar (.total is the count on the
+  // Dashboard tab); --line: one line of text, "2 emails · 1 WhatsApp message"
+  if (sub === "board") { const b = watchBoard(Math.min(168, Math.max(1, hours))); if (has("line")) return console.log(boardLine(b)); return out(b); }
   if (sub === "seen") return out(a1 ? seenWatch(a1) : { error: "Give the watch's id: symbiot watch board lists them." });
   if (sub === "new") return out(markNews(newsSince(hours), watchState().watches));
   // the same as the app's Draft a reply button: through the app when it runs, so its Agents tab tracks the run
@@ -422,6 +423,9 @@ async function cmdWatch() {
   symbiot watch board [--hours 24]             the Dashboard's cards: how many are new on
                                                each page, and the newest few (JSON, for an
                                                agent or a status bar: .total is the count)
+  symbiot watch board --line                   the same as one line of text, for a status
+                                               bar: "2 emails · 1 WhatsApp message" (an
+                                               empty line when nothing's new)
   symbiot watch seen <id>                      set a card's count back to 0, like its Seen
                                                button (what it found stays in watch new)
   symbiot watch check [id]                     read them now (JSON)