tryharness 0.7.22 → 0.7.23

package/README.md

@@ -50,15 +50,51 @@ command as soon as the agent reports its session id, including while the run is
 so the user can enter and steer the agent directly rather than interacting through a
 Harness wrapper.
 
-## Embedder
+## Local models
+
+Setup downloads the relay's core local models into `~/.harness/local`, each file checked
+against a pinned SHA-256 and every model under an allowlisted licence (`cli/lib/localmodels.mjs`):
+EmbeddingGemma 2's text part (one search space for code, agent sessions and board posts), an
+two injection guards (one reads long stretches and holds, one reads short ones and flags), a
+credential detector and a screenshot text reader (PP-OCR), plus llama.cpp's own server build
+(about 860 MB of models). Two more come down the first time they are used: speech for the buddy's
+Talk (Silero VAD, 1.3 MB, and on a Mac with macOS 26 Apple's own on-device transcriber through
+the buddy; elsewhere NVIDIA's Parakeet TDT 0.6B v3, 672 MB) and the search model's image and audio part
+(555 MB) when you first add a folder of recordings or screenshots. The search model runs on the
+GPU when this machine's check finds it faster, the CPU otherwise; the classifiers run in native
+ONNX Runtime. Background work runs at the OS's background priority and waits in Low Power Mode,
+on low battery and when the machine is hot.
+
+What they do:
+
+- **Briefs:** every brief a local coding agent gets carries a context pack (the files in its
+  repository most likely needed, earlier sessions on them, and a recording's minute or a
+  screenshot from your added folders when one is close) and the thread's task frame: what you
+  named resolved to file and line, the repository's state, other threads changing the same
+  repository, and each run's receipt. Agents can call `search_code`, `search_media`,
+  `compare_screens` (two screenshots of UI work, before and after: what changed and which text)
+  and `search_screen` (what you saw in tabs shared with Harness, asked of your own Harness window
+  sealed over the `tryharness link` line, so Harness's server carries it unread; your Allow in
+  that window comes first).
+- **Checks:** a finished run gets a receipt from its own session file; the coordinator hands an
+  untested "tests pass" to an agent to check. An agent's post that reads like orders to the
+  swarm is held; a key in anything sent to Harness's server goes as `[SECRET_n]` (and, in requests
+  that only decide, emails, phone numbers, wallet addresses and amounts as placeholders too); a
+  screenshot a cloud agent reads has keys, addresses and amounts covered; code a cloud agent asks to
+  see (`request_snippet`) leaves only when you share it (`tryharness board snippet`).
+- **Watches:** a command on this computer checked on a schedule (a heartbeat's age, a balance, a
+  count of error lines), said out loud only when it is off. Agents suggest them with
+  `suggest_watch`; nothing runs until you start it.
+- **Learning:** overnight, on the charger, the ranking model retrains on what the agents' runs
+  actually changed and is kept only if it beats the current one on the newest runs.
 
-Setup also downloads the embedding model the daemon uses for a task's context space,
-about 110 MB into `~/.harness/models`, reported like indexing is. It is the same model,
-quantization, pooling and normalization as Harness in the browser (bge-base, q8, CLS,
-L2), under the same version key, run on the wasm build of the ONNX runtime with no native
-addon, so a vector the daemon makes and one the browser makes describe the same space. The
-daemon never asks a browser tab to embed; until the model is present, context search
-answers from full-text search and says so.
+```sh
+npx tryharness local                  # what is installed, where it runs, what is indexed, the last learning
+npx tryharness local learn            # start the learner now
+npx tryharness local add ~/Recordings # a folder of recordings and screenshots agents can search
+npx tryharness watch                  # watches and what each last read
+npx tryharness watch add "keeper heartbeat" --run "cat state/heartbeat.json" --age completedAt --above 900 --every 5m
+```
 
 ## Task board
 
@@ -135,3 +171,7 @@ room. Agents read, post and search the room with `read_room`, `post_to_room` and
 Start. Harness facilitates and answers once. `tryharness goal show <id>` prints the room, `goal stop <id>` ends
 everything, `goal resume <id>` reopens a room that paused at a backstop, and `goal invite <id> handle/agent` asks another
 person's agent to join as a guest over a relay thread. `tryharness room post <goal> <text> [--guests]` posts as you.
+
+Parakeet TDT 0.6B v3 is by NVIDIA, licensed under CC BY 4.0
+(https://creativecommons.org/licenses/by/4.0/); the relay uses the 8-bit ONNX export by
+istupakov (https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx), unmodified.

package/assets/opencode-plugin.js

@@ -64,5 +64,6 @@ export const TryHarnessPlugin = async ({ directory, client }) => ({
     const messages = board ? await board.takeMessages(AGENT, sessionId) : [];
     const send = client?.session?.promptAsync?.bind(client.session) ?? client?.session?.prompt?.bind(client.session);
     if (messages.length && send) await send({ path: { id: sessionId }, body: { parts: [{ type: "text", text: board.messagesText(messages) }] } }).catch(() => {});
+    else if (board && !messages.length) await board.noteTurn(AGENT, { session_id: sessionId, cwd: directory }, pids);
   },
 });

package/assets/pi-extension.ts

@@ -149,6 +149,7 @@ export default function tryHarness(pi) {
     const board = await harness().catch(() => null);
     const text = board ? await said(board, ctx) : "";
     if (text) pi.sendUserMessage(text);
+    else if (board) await board.noteTurn(AGENT, where(ctx), pids);
   });
   // After each turn (pi 0.87 event name turn_end), only when the queue moved.
   pi.on("turn_end", async (_event, ctx) => {

package/cli.mjs

@@ -14,7 +14,7 @@ import { daemonRequest } from "./lib/control-client.mjs";
 import { ensureDir, readJson, writeJsonAtomic } from "./lib/files.mjs";
 import { paths } from "./lib/paths.mjs";
 import { currentRuntime, installRuntime } from "./lib/runtime-install.mjs";
-import { ensureModel, modelPresent } from "./lib/embedder.mjs";
+import { MODELS, ensureLlama, ensureModels, modelsPresent } from "./lib/localmodels.mjs";
 import { installService, serviceInfo, uninstallService } from "./lib/service.mjs";
 import { canShowDesktop, openDesktop, retireDesktop, setupDesktop } from "./lib/desktop.mjs";
 
@@ -173,14 +173,23 @@ async function connect() {
     const shown = await desktop();
     spin.stop(shown.text, shown.ok ? 0 : 2);
   }
-  // The context space's embedder lives with the daemon and comes down with setup, so the machine
-  // can search a task by meaning before the first task exists. About 110 MB, once.
-  if (modelPresent(paths.models)) ui.success("Embedder ready");
+  // The relay's core local models (localmodels.mjs) and the server that runs them come down with setup, so agents get
+  // context packs, code search, the handoff firewall and images checked before they leave from the first task. Each file
+  // is checked against its pinned hash.
+  const core = MODELS.filter((entry) => entry.tier === "core").map((entry) => entry.id);
+  const localRoot = path.join(paths.home, "local");
+  if (modelsPresent(localRoot, core)) ui.success("Local models ready");
   else {
     const download = ui.spinner();
-    download.start("Downloading the embedder (about 110 MB, once)");
-    try { await ensureModel({ modelsDir: paths.models, log: (line) => download.message(`Downloading the embedder · ${line.slice(0, 30)}`) }); download.stop("Embedder ready"); }
-    catch (error) { download.stop(`Embedder not downloaded (${error.message}). Task search uses words until the next setup run`, 2); }
+    const size = Math.round(MODELS.filter((entry) => core.includes(entry.id)).flatMap((entry) => entry.files).reduce((sum, file) => sum + file.size, 0) / 1e6);
+    download.start(`Downloading the local models (about ${size} MB, once)`);
+    // The classifiers run without the model server, so one that cannot start here does not hold them back.
+    const failures = [];
+    for (const step of [() => ensureLlama({ root: localRoot, log: (line) => download.message(`Downloading the local models · ${line.slice(0, 40)}`) }), () => ensureModels({ root: localRoot, ids: core, log: (line) => download.message(`Downloading the local models · ${line.slice(0, 40)}`) })]) {
+      try { await step(); } catch (error) { failures.push(error.message); }
+    }
+    if (failures.length) download.stop(`Local models not ready (${failures.join("; ")}). Agents work as before until the next setup run`, 2);
+    else download.stop("Local models ready");
   }
   const failed = states.filter((state) => selected.includes(state.agent) && state.installed && !state.configured);
   if (failed.length) {
@@ -227,6 +236,55 @@ async function status() {
   }
 }
 
+// The relay's local models: each one's job, licence, size and whether it is here, where they run, what the context engine
+// has indexed, and what the overnight learner last did. `tryharness local learn` starts the learner now; `local add
+// <folder>` and `local remove <folder>` name folders of recordings and screenshots for search_media and briefs.
+async function local() {
+  const root = path.join(paths.home, "local");
+  if (argv[0] === "add" || argv[0] === "remove") {
+    const response = await daemonRequest({ command: `local.${argv[0]}`, folder: path.resolve(argv[1] || process.cwd()) });
+    if (!response?.ok) throw new Error(response?.error || "the daemon did not answer");
+    console.log(argv[0] === "add" ? `Added. The relay reads the recordings and screenshots there in the background${response.downloading ? ", once the image and audio part of the search model has downloaded (555 MB)" : ""}.` : "Removed.");
+    return;
+  }
+  for (const entry of MODELS) {
+    const here = modelsPresent(root, [entry.id]);
+    const mb = Math.round(entry.files.reduce((sum, file) => sum + file.size, 0) / 1e6);
+    console.log(`${here ? "ready  " : "missing"}  ${entry.name} (${entry.tier}, ${entry.licence}, ${mb} MB): ${entry.purpose}`);
+  }
+  let current = null;
+  try { current = await daemonRequest({ command: argv[0] === "learn" ? "local.learn" : "local.status" }); }
+  catch (error) { console.log(`daemon: ${error.message}`); process.exitCode = 1; return; }
+  if (!current?.ok) { console.log(current?.error || "the daemon did not answer"); process.exitCode = 1; return; }
+  if (argv[0] === "learn") { console.log("Learning from your agents' runs now; tryharness local shows the result when it is done."); return; }
+  console.log(`runs on: ${current.device ? (current.device.gpu ? "GPU" : "CPU") : "not checked yet"}`);
+  for (const indexed of current.roots || []) console.log(`indexed: ${indexed.root} (${indexed.files} files)`);
+  if (current.pieces?.waiting) console.log(`waiting to be embedded: ${current.pieces.waiting} pieces`);
+  console.log(current.model ? `ranking: learned ${current.model.trainedAt}, held-out recall at 10 ${JSON.stringify(current.model.heldOut)}` : "ranking: the default until the learner has enough runs");
+  for (const folder of current.media?.folders || []) console.log(`recordings and screenshots: ${folder}`);
+  if (current.media?.folders?.length) console.log(`media files read: ${current.media.files}`);
+  if (current.learning) console.log(`last learning: ${current.learning.at} · ${current.learning.kept ? "kept a new model" : `kept the old one${current.learning.why ? ` (${current.learning.why})` : ""}`}`);
+}
+
+// Watches (watches.mjs): `tryharness watch` lists them and what each last read; `watch add` starts one; agents' suggestions
+// wait for `watch start`.
+async function watch() {
+  const sub = argv[0] || "list";
+  const minutes = (text) => { const found = String(text || "").match(/^(\d+(?:\.\d+)?)\s*(s|m|h)?$/); return found ? Number(found[1]) * ({ s: 1, m: 60, h: 3600 }[found[2] || "m"]) : null; };
+  const response = sub === "add" ? await daemonRequest({ command: "board.watch.add", agent: "user", name: argv[1], watchCommand: arg("--run", ""), folder: path.resolve(arg("--in", process.cwd())),
+    everySeconds: minutes(arg("--every", "5m")) ?? 300, field: arg("--field", undefined), age: arg("--age", undefined), count: arg("--count", undefined), above: arg("--above", undefined), below: arg("--below", undefined) })
+    : sub === "start" || sub === "remove" ? await daemonRequest({ command: `board.watch.${sub}`, agent: "user", id: argv[1] })
+    : await daemonRequest({ command: "board.watch.list", agent: "user" });
+  if (!response?.ok) throw new Error(response?.error || "the daemon did not answer");
+  if (sub === "add") { console.log(`Watching ${response.watch.name} every ${response.watch.everySeconds} s (${response.watch.id}).`); return; }
+  if (sub !== "list") { console.log("Done."); return; }
+  if (!response.watches.length) console.log("No watches. Add one: tryharness watch add \"keeper heartbeat\" --run \"<command>\" --age completedAt --above 900");
+  for (const entry of response.watches) {
+    const last = entry.samples.at(-1);
+    console.log(`${entry.state === "on" ? (entry.off ? "OFF  " : "ok   ") : "asks "} ${entry.name} · every ${entry.everySeconds} s · ${last ? (last.error ? `last check failed: ${last.error}` : `last read ${last.value} (${last.ms} ms)`) : "not checked yet"}${entry.state === "suggested" ? ` · suggested by ${entry.by}: tryharness watch start ${entry.id}` : ""}${entry.off ? ` · ${entry.off.text}` : ""}`);
+  }
+}
+
 async function doctor() {
   console.log(`tryharness ${VERSION}`);
   const config = readJson(paths.config, {});
@@ -341,6 +399,22 @@ async function board() {
     console.log(sub === "send" ? `Sent to ${response.message.recipient}.` : "Discarded.");
     return;
   }
+  // Code an agent elsewhere asked to see (snippets.mjs): list the requests waiting, or share or keep one.
+  if (sub === "snippet") {
+    const [id, choice] = argv.slice(argv.indexOf(sub) + 1);
+    if (!id) {
+      const response = await daemonRequest({ command: "board.snippet.list", agent: "user" });
+      if (!response?.ok) throw new Error(response?.error || "the daemon did not answer");
+      if (!response.snippets.length) { console.log("No agent is waiting to see code."); return; }
+      for (const each of response.snippets) console.log(`${each.id}  ${each.agent} asks to see ${each.path} lines ${each.from} to ${each.to} ("${each.about}")\n${each.text.split("\n").map((line) => `    ${line}`).join("\n")}\n  tryharness board snippet ${each.id} share|keep\n`);
+      return;
+    }
+    if (choice !== "share" && choice !== "keep") throw new Error("usage: tryharness board snippet <id> share|keep");
+    const response = await daemonRequest({ command: "board.snippet.decide", agent: "user", id, share: choice === "share" });
+    if (!response?.ok) throw new Error(response?.error || "the daemon did not answer");
+    console.log(choice === "share" ? "Shared in the thread." : "Kept on this computer.");
+    return;
+  }
   if (sub === "auto" || sub === "end") {
     const id = String(argv[argv.indexOf(sub) + 1] || "");
     const response = sub === "auto"
@@ -356,7 +430,7 @@ async function board() {
     if (!id || !query) throw new Error("usage: tryharness board context <task id> <what you are looking for>");
     const response = await daemonRequest({ command: "board.context.search", agent, id, query, limit: 8 }, 20_000);
     if (!response.ok) throw new Error(response.error || "search failed");
-    const how = response.retrieval === "vectors" ? "meaning" : response.embedder === "absent" ? "words (the embedder is not downloaded yet; run npx tryharness)" : `words (${response.context?.embedded ?? 0} of ${response.context?.total ?? 0} entries embedded so far)`;
+    const how = response.retrieval === "vectors" ? "meaning" : response.embedder === "absent" ? "words (the local models are not downloaded yet; run npx tryharness)" : `words (${response.context?.embedded ?? 0} of ${response.context?.total ?? 0} entries embedded so far)`;
     console.log(`${response.entries.length} result${response.entries.length === 1 ? "" : "s"} by ${how}`);
     for (const entry of response.entries) console.log(`\n[${entry.kind} · ${entry.author}${entry.source ? ` · ${entry.source}` : ""} · ${entry.score}]\n${entry.text.slice(0, 400)}`);
     return;
@@ -735,13 +809,15 @@ async function t3() {
 }
 
 function help() {
-  console.log(`tryharness ${VERSION}\n\nCommands:\n  tryharness                 set up everything and start the daemon\n  tryharness desktop         open the Harness desktop app (setup opens it too; --no-desktop skips it)\n  tryharness status          show daemon, bridge, agent, and index health\n  tryharness doctor          diagnose the complete connection\n  tryharness sync [--agent]  rescan local coding history in the background\n  tryharness task <id> --follow\n                             follow a delegated agent run in this terminal\n  tryharness board [--all]   list open work on the machine's task board\n  tryharness board post --for <agent> --objective <text> [--file <brief.md>]\n  tryharness board show <id>\n  tryharness board post --for codex --objective <text> --run --workspace <dir>\n                             the daemon runs it and reports back on the task\n  tryharness board run <id>  authorize a run an agent asked for\n  tryharness board release <id>\n  tryharness board agents    when each agent last read the board\n  tryharness board --watch   follow the board in this terminal\n  tryharness notify on|off   desktop notifications for what waits on you\n  tryharness board send <task> <message>   send a held reply to another person's agent\n  tryharness board auto <task> on|off      auto-send replies on that thread\n  tryharness board end <task> --outcome <text>\n  tryharness board context <task> <query>  search everything written about a task\n  tryharness goal set --objective <text> [--team a,b] [--workspace <dir>]   a goal your agents work together in one room\n  tryharness goal [list | show <id> | stop <id> | resume <id>]\n  tryharness room <goal> | room post <goal> <text> [--ask agent,agent]\n  tryharness swarm rounds [<thread>]   what the coordinator saw, chose and cost each round\n  tryharness swarm coordinator [harness|local]   who coordinates your threads\n  tryharness connect <handle/agent> --about <text>   ask another person's agent to work an issue with yours, sealed to their computer\n  tryharness peers           the keys of other people's computers this one has worked with, to compare\n  tryharness agents          how Harness wakes each of your agents, and set up the rest\n  tryharness agents setup <agent> | agents test <agent>\n  tryharness agents cards     what each of your agents is told about the others\n  tryharness agents note <agent> <text> [--clear]   your own note on an agent's card, told to every other agent\n  tryharness agents models [<agent>] [--refresh]   the models each coding agent lists and the one it runs on\n  tryharness agents model <agent> <model> [--effort <level>] | auto | default   the model it runs on; auto lets Harness pick per run\n  tryharness agents routing "<text>" [--clear]   how Auto picks models for your swarm\n  tryharness mcp grok-bot    print the MCP snippet that puts Grok Bot on the board\n  tryharness link            connect the browser you use Harness in to this computer, end to end (--print shows the link)\n  tryharness phone pair      show a QR code that pairs your phone with this computer\n  tryharness phone list | revoke <id>\n  tryharness t3              approve Harness in T3 Code once, so your messages go into its threads (t3 status | t3 forget)\n  tryharness logs [--follow]\n  tryharness disconnect\n`);
+  console.log(`tryharness ${VERSION}\n\nCommands:\n  tryharness                 set up everything and start the daemon\n  tryharness desktop         open the Harness desktop app (setup opens it too; --no-desktop skips it)\n  tryharness status          show daemon, bridge, agent, and index health\n  tryharness doctor          diagnose the complete connection\n  tryharness local [learn]   the relay's local models, where they run, what is indexed and the last learning\n  tryharness local add|remove <folder>   a folder of recordings and screenshots agents can search by what was said or shown\n  tryharness watch [add <name> --run <command> [--every 5m] [--field f | --age f | --count text] [--above n] [--below n] | start <id> | remove <id>]\n                             watches the relay checks and says out loud only when they are off\n  tryharness sync [--agent]  rescan local coding history in the background\n  tryharness task <id> --follow\n                             follow a delegated agent run in this terminal\n  tryharness board [--all]   list open work on the machine's task board\n  tryharness board post --for <agent> --objective <text> [--file <brief.md>]\n  tryharness board show <id>\n  tryharness board post --for codex --objective <text> --run --workspace <dir>\n                             the daemon runs it and reports back on the task\n  tryharness board run <id>  authorize a run an agent asked for\n  tryharness board release <id>\n  tryharness board agents    when each agent last read the board\n  tryharness board --watch   follow the board in this terminal\n  tryharness notify on|off   desktop notifications for what waits on you\n  tryharness board send <task> <message>   send a held reply to another person's agent\n  tryharness board snippet [<id> share|keep]   code an agent elsewhere asked to see; nothing leaves until you share it\n  tryharness board auto <task> on|off      auto-send replies on that thread\n  tryharness board end <task> --outcome <text>\n  tryharness board context <task> <query>  search everything written about a task\n  tryharness goal set --objective <text> [--team a,b] [--workspace <dir>]   a goal your agents work together in one room\n  tryharness goal [list | show <id> | stop <id> | resume <id>]\n  tryharness room <goal> | room post <goal> <text> [--ask agent,agent]\n  tryharness swarm rounds [<thread>]   what the coordinator saw, chose and cost each round\n  tryharness swarm coordinator [harness|local]   who coordinates your threads\n  tryharness connect <handle/agent> --about <text>   ask another person's agent to work an issue with yours, sealed to their computer\n  tryharness peers           the keys of other people's computers this one has worked with, to compare\n  tryharness agents          how Harness wakes each of your agents, and set up the rest\n  tryharness agents setup <agent> | agents test <agent>\n  tryharness agents cards     what each of your agents is told about the others\n  tryharness agents note <agent> <text> [--clear]   your own note on an agent's card, told to every other agent\n  tryharness agents models [<agent>] [--refresh]   the models each coding agent lists and the one it runs on\n  tryharness agents model <agent> <model> [--effort <level>] | auto | default   the model it runs on; auto lets Harness pick per run\n  tryharness agents routing "<text>" [--clear]   how Auto picks models for your swarm\n  tryharness mcp grok-bot    print the MCP snippet that puts Grok Bot on the board\n  tryharness link            connect the browser you use Harness in to this computer, end to end (--print shows the link)\n  tryharness phone pair      show a QR code that pairs your phone with this computer\n  tryharness phone list | revoke <id>\n  tryharness t3              approve Harness in T3 Code once, so your messages go into its threads (t3 status | t3 forget)\n  tryharness logs [--follow]\n  tryharness disconnect\n`);
 }
 
 try {
   if (command === "setup" || (command === "connect" && (!argv[0] || argv[0].startsWith("-")))) await connect();
   else if (command === "status" || command === "bridge") await status();
   else if (command === "doctor") await doctor();
+  else if (command === "local") await local();
+  else if (command === "watch") await watch();
   else if (command === "sync" || command === "backfill" || command === "drain") await sync();
   else if (command === "task") await task();
   else if (command === "board") await board();

package/lib/atlas.mjs

@@ -0,0 +1,304 @@
+import fs from "node:fs";
+import path from "node:path";
+import { createHash } from "node:crypto";
+import { spawnSync } from "node:child_process";
+import sqliteWasm from "node-sqlite3-wasm";
+import { fromBlob, toBlob } from "./vectors.mjs";
+
+// The relay's local index of the code its agents work in: every text file in the folders the swarm uses, cut into
+// pieces at top-level declarations and headings, each piece embedded once by the search model and found again by words
+// (SQLite's full-text index) and by meaning. Pieces are keyed by their repository path and text, not by commit, so a
+// new worktree or a small edit costs only the pieces that changed. Files that look like keys or secrets are never read.
+
+const { Database } = sqliteWasm;
+export const ATLAS_VERSION = "embeddinggemma-2:q8:768:v1";
+const MAX_FILE = 300_000;
+const MIN_PIECE = 300;
+const MAX_PIECE = 2_400;
+const TEXT_EXT = /\.(?:[cm]?[jt]sx?|json|md|mdx|txt|ya?ml|toml|sql|sol|rs|go|py|rb|java|kt|swift|c|cc|cpp|h|hpp|cs|php|sh|zsh|css|scss|html|vue|svelte|astro|graphql|proto|ini|cfg|conf|env\.example|lua|ex|exs|dart|scala|zig)$/i;
+const SKIP_PATH = /(?:^|\/)(?:node_modules|dist|build|out|\.next|coverage|vendor|target|\.git)\/|(?:^|\/)(?:package-lock\.json|pnpm-lock\.yaml|yarn\.lock|Cargo\.lock|go\.sum)$|\.min\.[jc]ss?$|\.map$/;
+const SECRET_PATH = /(?:^|\/)(?:\.env(?:\.[\w-]+)?|id_[rd]sa\w*|[\w.-]+\.(?:pem|key|p12|pfx|keystore|jks)|\.npmrc|\.pypirc|credentials(?:\.json)?)$/i;
+const SECRET_TEXT = /-----BEGIN (?:[A-Z]+ )?PRIVATE KEY-----/;
+const DECLARATION = /^(?:export\s|import\s|async\s|function\s|class\s|const\s|let\s|var\s|type\s|interface\s|enum\s|def\s|fn\s|pub\s|impl\s|struct\s|mod\s|func\s|package\s|contract\s|library\s|module\s|describe\(|it\(|test\(|#{1,6}\s|\[[\w.-]+\]$|create\s|alter\s)/i;
+const STOP = new Set("the a an and or of to in on for with is are be it this that from by at as was were can should would could will do does did not no yes we you i me my our your they them their its into out up so if then else than but about how what why when where which who whom also just like make made use used using need needs want".split(" "));
+
+/** Whether a repository path may be indexed: a text file, not generated, not a secret. */
+export const indexable = (rel, size = 0) => TEXT_EXT.test(rel) && !SKIP_PATH.test(rel) && !SECRET_PATH.test(rel) && size <= MAX_FILE;
+
+/** A file's pieces: { text, line } cut at top-level declarations and headings, small ones merged, big ones split at blank lines. */
+export function pieces(text) {
+  const lines = text.split("\n");
+  const segments = [];
+  let start = 0;
+  for (let i = 1; i <= lines.length; i += 1) {
+    const line = lines[i];
+    const boundary = i === lines.length || (line && !/^\s/.test(line) && (lines[i - 1].trim() === "" || DECLARATION.test(line)) && !/^[)}\]]/.test(line));
+    if (boundary) { segments.push({ line: start + 1, text: lines.slice(start, i).join("\n") }); start = i; }
+  }
+  const merged = [];
+  for (const segment of segments) {
+    const last = merged.at(-1);
+    if (last && last.text.length < MIN_PIECE) { last.text += `\n${segment.text}`; continue; }
+    merged.push({ ...segment });
+  }
+  const out = [];
+  for (const segment of merged) {
+    if (!segment.text.trim()) continue;
+    if (segment.text.length <= MAX_PIECE) { out.push(segment); continue; }
+    let buffer = "";
+    let line = segment.line;
+    let at = segment.line;
+    for (const block of segment.text.split(/\n(?=\s*\n)/)) {
+      if (buffer && buffer.length + block.length > MAX_PIECE) { out.push({ line, text: buffer }); buffer = ""; line = at; }
+      for (let i = 0; i < block.length; i += MAX_PIECE) {
+        const part = block.slice(i, i + MAX_PIECE);
+        if (buffer && buffer.length + part.length > MAX_PIECE) { out.push({ line, text: buffer }); buffer = ""; line = at; }
+        buffer += (buffer ? "\n" : "") + part;
+      }
+      at += block.split("\n").length;
+    }
+    if (buffer.trim()) out.push({ line, text: buffer });
+  }
+  return out;
+}
+
+export const pieceHash = (rel, text) => createHash("sha1").update(`${rel}\0${text}`).digest("hex");
+
+/** Words and identifiers worth searching for in a request, most specific first (identifiers, paths, then words). */
+export function queryTerms(text, max = 24) {
+  const raw = String(text).match(/[\p{L}\p{N}_$./-]{3,}/gu) || [];
+  const terms = [];
+  for (const token of raw) {
+    const clean = token.replace(/^[./-]+|[./-]+$/g, "");
+    if (clean.length < 3 || STOP.has(clean.toLowerCase())) continue;
+    for (const part of clean.split(/[./-]/)) if (part.length >= 3 && !STOP.has(part.toLowerCase())) terms.push(part);
+  }
+  const specific = (term) => (/[A-Z_$]/.test(term.slice(1)) || /\d/.test(term) ? 0 : 1);
+  return [...new Set(terms)].sort((a, b) => specific(a) - specific(b)).slice(0, max);
+}
+
+export function openAtlas(file) {
+  const db = new Database(file);
+  // The atlas is a cache the relay can always rebuild from the repositories and session files, so writes skip the disk sync.
+  db.exec("pragma journal_mode = MEMORY; pragma synchronous = OFF; pragma busy_timeout = 5000;");
+  db.exec(`
+    create table if not exists pieces (hash text primary key, path text not null, line integer not null, text text not null);
+    create table if not exists piece_vectors (hash text not null, version text not null, vector blob not null, primary key(hash, version));
+    create table if not exists blobs (blob text primary key, path text not null, pieces text not null);
+    create table if not exists files (root text not null, path text not null, blob text not null, indexed_at text not null, primary key(root, path));
+    create virtual table if not exists pieces_fts using fts5(hash unindexed, path, text, tokenize='unicode61 tokenchars ''_$''');
+    create table if not exists runs (agent text not null, session_id text not null, root text not null, at integer not null, brief text not null,
+      edited text not null, read text not null, before text not null, source_mtime real not null, primary key(agent, session_id));
+    create index if not exists runs_root_at on runs(root, at);
+    create table if not exists run_vectors (agent text not null, session_id text not null, version text not null, vector blob not null, primary key(agent, session_id, version));
+    create table if not exists meta (key text primary key, value text not null);
+    create table if not exists media (hash text primary key, path text not null, kind text not null, start_ms integer not null, end_ms integer not null, mtime real not null);
+    create index if not exists media_path on media(path);
+  `);
+  // A blob's pieces never change, so lookups are kept (a few MB for a large repository and its recent versions).
+  const blobPieces = new Map();
+  const run = (sql, ...values) => db.run(sql, values.length ? values : undefined);
+  const all = (sql, ...values) => db.all(sql, values.length ? values : undefined);
+  const get = (sql, ...values) => db.get(sql, values.length ? values : undefined);
+  return {
+    close: () => db.close(),
+    blob(blob) {
+      if (blobPieces.has(blob)) return blobPieces.get(blob);
+      const row = get("select pieces from blobs where blob = ?", blob);
+      if (!row) return null;
+      if (blobPieces.size >= 200_000) blobPieces.clear();
+      blobPieces.set(blob, JSON.parse(row.pieces));
+      return blobPieces.get(blob);
+    },
+    /** Runs fn inside one transaction. */
+    batch(fn) {
+      db.exec("begin");
+      try { const out = fn(); db.exec("commit"); return out; } catch (error) { db.exec("rollback"); throw error; }
+    },
+    putBlob(blob, rel, parts) {
+      const hashes = [];
+      for (const part of parts) {
+        const hash = pieceHash(rel, part.text);
+        hashes.push(hash);
+        if (!get("select 1 as one from pieces where hash = ?", hash)) {
+          run("insert into pieces(hash, path, line, text) values(?, ?, ?, ?)", hash, rel, part.line, part.text);
+          run("insert into pieces_fts(hash, path, text) values(?, ?, ?)", hash, rel.replace(/[/._-]/g, " "), part.text);
+        }
+      }
+      run("insert or replace into blobs(blob, path, pieces) values(?, ?, ?)", blob, rel, JSON.stringify(hashes));
+      return hashes;
+    },
+    setFiles(root, rows) {
+      db.exec("begin");
+      try {
+        run("delete from files where root = ?", root);
+        const now = new Date().toISOString();
+        for (const row of rows) run("insert into files(root, path, blob, indexed_at) values(?, ?, ?, ?)", root, row.path, row.blob, now);
+        db.exec("commit");
+      } catch (error) { db.exec("rollback"); throw error; }
+    },
+    files: (root) => all("select path, blob from files where root = ?", root),
+    roots: () => all("select root, count(*) as files, max(indexed_at) as indexed_at from files group by root").map((row) => ({ ...row, files: Number(row.files) })),
+    toEmbed: (version, limit = 64) => all("select p.hash, p.path, p.text from pieces p left join piece_vectors v on v.hash = p.hash and v.version = ? where v.hash is null limit ?", version, limit),
+    missing: (version) => Number(get("select count(*) as n from pieces p left join piece_vectors v on v.hash = p.hash and v.version = ? where v.hash is null", version).n),
+    putVector: (hash, version, vector) => run("insert or replace into piece_vectors(hash, version, vector) values(?, ?, ?)", hash, version, toBlob(vector)),
+    vectors(hashes, version) {
+      const out = new Map();
+      const list = [...hashes];
+      for (let i = 0; i < list.length; i += 500) {
+        const slice = list.slice(i, i + 500);
+        for (const row of all(`select hash, vector from piece_vectors where version = ? and hash in (${slice.map(() => "?").join(",")})`, version, ...slice)) out.set(row.hash, fromBlob(row.vector));
+      }
+      return out;
+    },
+    piece: (hash) => get("select hash, path, line, text from pieces where hash = ?", hash),
+    meta(key, value) {
+      if (value === undefined) return get("select value from meta where key = ?", key)?.value ?? null;
+      run("insert or replace into meta(key, value) values(?, ?)", key, String(value));
+      return value;
+    },
+    runSeen: (agent, sessionId) => get("select source_mtime from runs where agent = ? and session_id = ?", agent, sessionId)?.source_mtime ?? null,
+    putRun(row) {
+      run(`insert or replace into runs(agent, session_id, root, at, brief, edited, read, before, source_mtime) values(?, ?, ?, ?, ?, ?, ?, ?, ?)`,
+        row.agent, row.sessionId, row.root, row.at, row.brief.slice(0, 4_000), JSON.stringify(row.edited), JSON.stringify(row.read), JSON.stringify(row.before), row.sourceMtime);
+    },
+    runs: (root, since = 0) => all("select agent, session_id, root, at, brief, edited, read, before from runs where root = ? and at >= ? order by at", root, since)
+      .map((row) => ({ ...row, at: Number(row.at), edited: JSON.parse(row.edited), read: JSON.parse(row.read), before: JSON.parse(row.before) })),
+    runsToEmbed: (version, limit = 32) => all("select r.agent, r.session_id, r.brief from runs r left join run_vectors v on v.agent = r.agent and v.session_id = r.session_id and v.version = ? where v.agent is null limit ?", version, limit),
+    putRunVector: (agent, sessionId, version, vector) => run("insert or replace into run_vectors(agent, session_id, version, vector) values(?, ?, ?, ?)", agent, sessionId, version, toBlob(vector)),
+    runVectors: (root, version) => all("select r.agent, r.session_id, r.at, r.brief, r.edited, v.vector from runs r join run_vectors v on v.agent = r.agent and v.session_id = r.session_id and v.version = ? where r.root = ?", version, root)
+      .map((row) => ({ agent: row.agent, sessionId: row.session_id, at: Number(row.at), brief: row.brief, edited: JSON.parse(row.edited), vector: fromBlob(row.vector) })),
+    /** Recordings' windows and images (media.mjs): one row each, its vector beside the pieces'. */
+    mediaSeen: (file) => get("select max(mtime) as mtime from media where path = ?", file)?.mtime ?? null,
+    mediaPaths: () => all("select distinct path from media").map((row) => row.path),
+    dropMedia: (file) => run("delete from media where path = ?", file),
+    putMedia: (row) => run("insert or replace into media(hash, path, kind, start_ms, end_ms, mtime) values(?, ?, ?, ?, ?, ?)", row.hash, row.path, row.kind, row.startMs, row.endMs, row.mtime),
+    mediaVectors: (version) => all("select m.hash, m.path, m.kind, m.start_ms, m.end_ms, v.vector from media m join piece_vectors v on v.hash = m.hash and v.version = ?", version)
+      .map((row) => ({ hash: row.hash, path: row.path, kind: row.kind, startMs: Number(row.start_ms), endMs: Number(row.end_ms), vector: fromBlob(row.vector) })),
+    /** Full-text matches: { hash, rank } best first (bm25, lower is better), over every indexed piece. */
+    words(terms, limit = 400) {
+      if (!terms.length) return [];
+      const match = terms.map((term) => `"${term.replaceAll('"', '""')}"`).join(" OR ");
+      return all("select hash, bm25(pieces_fts, 0, 2.0, 1.0) as rank from pieces_fts where pieces_fts match ? order by rank limit ?", match, limit);
+    },
+  };
+}
+
+const git = (cwd, args, input) => spawnSync("git", args, { cwd, input, encoding: "utf8", maxBuffer: 256 * 2 ** 20 });
+
+/** The repository's files at a commit (or the working tree when commit is null): [{ path, blob, size }]. */
+export function treeFiles(root, commit = null) {
+  if (commit) {
+    const listed = git(root, ["ls-tree", "-r", "-l", commit]);
+    if (listed.status !== 0) return [];
+    return listed.stdout.split("\n").filter(Boolean).map((line) => {
+      const [meta, rel] = line.split("\t");
+      const [, type, blob, size] = meta.split(/\s+/);
+      return type === "blob" ? { path: rel, blob, size: Number(size) || 0 } : null;
+    }).filter((row) => row && indexable(row.path, row.size));
+  }
+  // The working tree: git's index already holds the blob of every unchanged tracked file; only files changed on disk
+  // and new files git does not ignore are hashed, so a pass over a large repository costs about what `git status` does.
+  const staged = git(root, ["ls-files", "-s", "-z"]);
+  if (staged.status !== 0) return [];
+  const blobs = new Map();
+  for (const entry of staged.stdout.split("\0")) {
+    const tab = entry.indexOf("\t");
+    if (tab < 0) continue;
+    const [, blob, stage] = entry.slice(0, tab).split(" ");
+    if (stage === "0") blobs.set(entry.slice(tab + 1), blob);
+  }
+  const changed = git(root, ["diff", "--name-only", "-z"]).stdout.split("\0").filter(Boolean);
+  const untracked = git(root, ["ls-files", "--others", "--exclude-standard", "-z"]).stdout.split("\0").filter(Boolean);
+  const rows = [];
+  const rehash = [];
+  for (const rel of new Set([...blobs.keys(), ...untracked])) {
+    if (!indexable(rel, 0)) continue;
+    let stat;
+    try { stat = fs.statSync(path.join(root, rel)); } catch { continue; }
+    if (!stat.isFile() || stat.size > MAX_FILE) continue;
+    const row = { path: rel, size: stat.size, blob: blobs.get(rel) };
+    rows.push(row);
+    if (!row.blob || changed.includes(rel)) rehash.push(row);
+  }
+  if (rehash.length) {
+    const hashed = git(root, ["hash-object", "--stdin-paths"], rehash.map((row) => path.join(root, row.path)).join("\n"));
+    const out = hashed.status === 0 ? hashed.stdout.split("\n") : [];
+    rehash.forEach((row, i) => { row.blob = out[i]; });
+  }
+  return rows.filter((row) => /^[0-9a-f]{40}$/.test(row.blob || ""));
+}
+
+/** File contents for blobs, read through one `git cat-file --batch`. */
+export function readBlobs(root, blobs) {
+  const out = new Map();
+  if (!blobs.length) return out;
+  const result = spawnSync("git", ["cat-file", "--batch"], { cwd: root, input: blobs.join("\n") + "\n", maxBuffer: 2 * 2 ** 30 });
+  const buffer = result.stdout;
+  let at = 0;
+  while (at < buffer.length) {
+    const end = buffer.indexOf(10, at);
+    if (end < 0) break;
+    const [blob, type, size] = buffer.subarray(at, end).toString("utf8").split(" ");
+    if (type === "missing" || !size) { at = end + 1; continue; }
+    const length = Number(size);
+    const body = buffer.subarray(end + 1, end + 1 + length);
+    at = end + 1 + length + 1;
+    if (!body.includes(0) && !SECRET_TEXT.test(body.toString("latin1"))) out.set(blob, body.toString("utf8"));
+  }
+  return out;
+}
+
+/** Adds the files' pieces to the atlas (new blobs only); returns how many blobs were new. */
+export function addFiles(atlas, root, rows) {
+  const fresh = rows.filter((row) => !atlas.blob(row.blob));
+  const texts = readBlobs(root, [...new Set(fresh.map((row) => row.blob))]);
+  for (let i = 0; i < fresh.length; i += 500) atlas.batch(() => {
+    for (const row of fresh.slice(i, i + 500)) {
+      const text = texts.get(row.blob);
+      atlas.putBlob(row.blob, row.path, text === undefined ? [] : pieces(text));
+    }
+  });
+  return fresh.length;
+}
+
+/** Embeds every piece that has no vector yet, in batches, on the given lane. */
+export async function embedMissing(atlas, inference, { lane = "background", version = ATLAS_VERSION, batch = 32, onProgress } = {}) {
+  let done = 0;
+  for (;;) {
+    const rows = atlas.toEmbed(version, batch);
+    if (!rows.length) return done;
+    const vectors = await inference.embed(rows.map((row) => ({ kind: "document", title: row.path, text: row.text })), { lane });
+    atlas.batch(() => rows.forEach((row, i) => atlas.putVector(row.hash, version, vectors[i])));
+    done += rows.length;
+    onProgress?.(done);
+  }
+}
+
+/**
+ * Files in scope ranked for a request, with the signals each one scored on. scope: [{ path, blob }]. Returns
+ * [{ path, blob, dense, lexical, named, best: { hash, line } }] sorted by the given scorer (default: dense+lexical+named).
+ */
+export async function searchFiles(atlas, inference, query, scope, { version = ATLAS_VERSION, lane = "interactive", queryVector = null, vectors = null, wordLimit = 400 } = {}) {
+  const byHash = new Map();
+  const files = scope.map((row) => ({ path: row.path, blob: row.blob, hashes: atlas.blob(row.blob) ?? [] }));
+  for (const file of files) for (const hash of file.hashes) { if (!byHash.has(hash)) byHash.set(hash, []); byHash.get(hash).push(file); }
+  const qv = queryVector ?? (await inference.embed([{ kind: "code-query", text: String(query).slice(0, 6_000) }], { lane }))[0];
+  const pieceVectors = vectors ?? atlas.vectors(byHash.keys(), version);
+  const terms = queryTerms(query);
+  const lexical = new Map();
+  for (const row of atlas.words(terms, wordLimit)) if (byHash.has(row.hash)) lexical.set(row.hash, -row.rank);
+  const lowered = String(query).toLowerCase();
+  return files.map((file) => {
+    let dense = -1, best = null, words = 0;
+    for (const hash of file.hashes) {
+      const vector = pieceVectors.get(hash);
+      if (vector) { let dot = 0; for (let i = 0; i < qv.length; i += 1) dot += qv[i] * vector[i]; if (dot > dense) { dense = dot; best = hash; } }
+      if (lexical.has(hash)) words = Math.max(words, lexical.get(hash));
+    }
+    const base = path.basename(file.path).toLowerCase();
+    const stem = base.replace(/\.[^.]+$/, "");
+    const named = lowered.includes(file.path.toLowerCase()) ? 2 : lowered.includes(base) ? 1.5 : stem.length > 3 && new RegExp(`\\b${stem.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}\\b`).test(lowered) ? 1 : 0;
+    return { path: file.path, blob: file.blob, dense, lexical: words, named, best };
+  });
+}