canon-atlas 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Shane Conner
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,103 @@
+# canon-atlas
+
+An atlas over a catalog of markdown documents that reference each other. Point it at a directory and it draws the whole corpus as a constellation: documents as lit spheres, references as gossamer links, semantic clusters as colored regions of the map. There is no chrome beyond what the map needs: an icon rail on the left owns the panels (documents with search and ordering, the map's clusters and key, and in the app your folders), and a reader on the right opens a tab per document, with editing once a folder or a server backs the page. Every document you open is a browser history entry, so back and forward walk the trail of articles you read, and a hierarchical address renders as breadcrumbs above the document.
+
+It is agnostic to what wrote the catalog. It was built as the human viewport onto [pi-canon](https://github.com/shaneconner/pi-canon) project memory, where mutable articles carry current ground truth and an immutable journal carries the journey to it, but any folder of markdown with `[[wikilinks]]` or relative links works: an Obsidian vault, a wiki, a notes directory, an agent memory store.
+
+![The constellation over a pi-canon memory store](docs/constellation.png)
+
+## Quick start
+
+```sh
+npx canon-atlas open           # open the atlas in your browser; pick folders in the page
+npx canon-atlas embed .        # write embeddings.json, so color means meaning
+npx canon-atlas build .        # write atlas.html, one self-contained chart of this folder
+npx canon-atlas serve .        # serve the live atlas at http://127.0.0.1:4747/
+npx canon-atlas app            # write atlas-app.html, the same app as one file
+```
+
+Four ways into the same page.
+
+- **open** is the everyday door, made for a launcher entry: it starts the little localhost host if one is not already running, opens your browser at it, and gets out of the way. Type a project's absolute path in the page and the host reads the corpus itself, remembers the path, and lists it in the folders panel as exactly that path, so ten projects whose stores are all named `.canon` still tell apart at a glance. Picking or dropping a folder works too; every project lives in its own browser tab off the one host. Atlas tabs quietly ping the host, and after a full day with no atlas tab open it exits itself, so nothing lingers. A desktop entry makes it a one-keystroke launch:
+
+  ```ini
+  [Desktop Entry]
+  Type=Application
+  Name=canon-atlas
+  Exec=canon-atlas open
+  Terminal=false
+  ```
+
+- **app** emits the pure-client app itself: one HTML file with the renderer and the whole build pipeline inlined, and no data of its own. Host it anywhere static, or open it straight from disk. Editing uses the File System Access API, which needs Chrome or Edge and an http or https page: browsers block the writable picker on `file://`, and some browsers (Brave, Firefox) ship without the API (Brave re-enables it at `brave://flags/#file-system-access-api`). Everywhere else the page falls back to a read-only folder picker, and a folder dropped anywhere on the page opens too, so the constellation still works from a double-clicked file. Remembered folders label themselves from the corpus (the picker API never reveals paths), and any label can be renamed in the folders panel; a name you type, like the full path, is kept as written. When the page comes from the local `open` host, the path door above does better: the host knows real paths, and its workspaces need no picker at all.
+- **build** emits a chart: a single HTML file with the renderer, the graph, and every document inlined. It opens from `file://`, ships as an email attachment, and publishes as a static page. Read only by nature.
+- **serve** binds to localhost and puts the corpus behind the same page: create, edit, and delete from the reader panel, with changes on disk immediately. Immutable collections are append-only, so the tool will create a journal entry but never rewrite or delete one.
+
+The **open**, **app**, and **serve** paths build the wire data from the exact same pipeline, so the constellation is identical whether the markdown is read in the browser or by the server.
+
+## Collections
+
+A corpus is described by collections: named groups of documents with their own frontmatter mapping and their own mutability. Without configuration:
+
+- A root holding `articles/` and `journal/` gets the **pi-canon preset**: articles are mutable, the journal is immutable and append-only, `capsule` becomes the summary, a journal entry's `subject` list becomes edges to the articles it concerns, and tags of the form `path:VALUE`, which a migrated store carries as provenance, become source metadata instead of tags.
+- Anything else is read as a single mutable collection of notes.
+
+To define your own, put a `canon-atlas.json` at the root:
+
+```json
+{
+  "title": "team wiki",
+  "collections": [
+    { "name": "pages", "match": "wiki/" },
+    { "name": "decisions", "match": "wiki/adr/", "immutable": true,
+      "fields": { "summary": "status", "date": "decided" } }
+  ]
+}
+```
+
+`match` is a directory prefix; the longest prefix claims a file. `fields` maps your frontmatter keys onto the ones the atlas reads: `title`, `tags`, `date`, `summary`, and `refs` (a frontmatter list of documents this one concerns, rendered as edges). Titles fall back to the first heading, then to the path. `immutable` marks a collection append-only and draws its documents as rings instead of spheres.
+
+`reveal` controls whether a collection starts on the map: `"always"` (the default) or `"off"` (`"focus"` is accepted and means the same). A collection is on the map exactly when its row in the map panel's key is on: off means gone, layout included, so a hidden tier claims no empty space and nothing lingers. The pi-canon preset starts the journal off: a hundred immutable event entries are detail on demand, not weather. This is why a large corpus reads as a calm sky.
+
+References that resolve nowhere are not errors: they render as dashed, unwritten nodes, because a name the corpus reaches for but has not written yet is a fact worth seeing. In live mode an unwritten node offers to be written.
+
+## Color
+
+Color is the cluster a document belongs to, sampled from one perceptual ramp (Batlow), and the force layout pulls each cluster into its own region, so a band of the palette and a region of the map mean the same thing.
+
+Clusters come from the best signal available:
+
+- **Embeddings**, when the root holds an `embeddings.json` mapping each document path to a vector. Spherical k-means picks the cluster count by silhouette. Bring vectors from any model; the file is the interface.
+
+  ```json
+  { "articles/core.md": [0.021, -0.113, ...], "articles/helper.md": [...] }
+  ```
+
+  `canon-atlas embed [root]` writes that file for you. It is the one command in the atlas that touches a network, and only the provider named: the default is a local [Ollama](https://ollama.com) (`nomic-embed-text`; `OLLAMA_HOST` overrides the address), so by default nothing leaves the machine. `-m openai` or `-m openai:MODEL` uses the OpenAI API instead, reading `OPENAI_API_KEY` from the environment. Reruns embed only documents whose content changed, drop vectors for deleted documents, and write atomically after every batch, so an interrupted run keeps its progress. A corpus embedded once keeps its recorded model on bare reruns; `-m` switches models and re-embeds everything. A small `__meta__` record inside the file carries the model name and content hashes; readers ignore it.
+
+- **Paths**, when addresses form a real hierarchy (`src/core/config`, `meta/memory/folds/x`): documents cluster by the first path segment, labeled by the segment itself. A hierarchical store names its own neighborhoods.
+
+- **Link structure** otherwise: deterministic greedy modularity over the reference graph, at zero cost, fully offline.
+
+The map panel can also color by collection or by tag.
+
+Labels reveal progressively with zoom: the most central documents keep their names at any distance, and the rest earn them as you approach.
+
+## Vocabulary
+
+The tool is the **atlas**. The graph view is the **constellation**. The self-contained file the build emits is a **chart**. The serverless page that opens folders in the browser is the **app**. A star atlas is a book of constellation maps, and in mathematics an atlas is a collection of charts that together cover a space; both senses are meant.
+
+## Design constraints
+
+- No runtime dependencies. d3 is vendored (ISC, see `vendor/LICENSE.d3`), everything else is the platform.
+- The chart is one file. The only external reference is the Google Fonts stylesheet, and the page falls back to system fonts without it.
+- The server binds to 127.0.0.1 only, refuses any path outside the corpus, refuses any path that crosses a symbolic link, and enforces collection mutability at the API, not just in the UI. Because the page inlines the whole corpus, the editing API also requires a loopback Host and a same-origin, JSON request, so a web page on another origin cannot drive it by DNS rebinding or a cross-site POST. The launcher host's workspace registry sits behind the same guards, and every registered workspace is served by the same handlers.
+- Rendered markdown is escaped before anything else touches it; a document cannot inject markup into the page.
+- The gate suite is the contract: `node tests/verify.mjs`, every gate green before anything lands.
+
+## Provenance
+
+The constellation rendering is a port of the graph view the author built for the pi-canon project pages, itself descended from a knowledge-graph viewer built for an internal agent harness. The articles and journal framing, one mutable tier of ground truth over one immutable tier of events, comes from pi-canon; the atlas only asks that your corpus be markdown files that name each other.
+
+## License
+
+MIT

package/package.json

@@ -0,0 +1,46 @@
+{
+  "name": "canon-atlas",
+  "version": "0.1.0",
+  "description": "An atlas over a catalog of markdown documents that reference each other: a constellation graph, search, a reader, and local editing. Agnostic to what wrote the catalog.",
+  "type": "module",
+  "bin": {
+    "canon-atlas": "src/cli.mjs"
+  },
+  "exports": {
+    ".": "./src/build.mjs",
+    "./package.json": "./package.json"
+  },
+  "files": [
+    "src",
+    "vendor",
+    "README.md",
+    "LICENSE"
+  ],
+  "engines": {
+    "node": ">=20"
+  },
+  "scripts": {
+    "test": "node tests/verify.mjs"
+  },
+  "keywords": [
+    "knowledge-graph",
+    "constellation",
+    "markdown",
+    "wiki",
+    "memory",
+    "agent-memory",
+    "llm-wiki",
+    "visualization",
+    "d3",
+    "pi-canon",
+    "obsidian"
+  ],
+  "author": "Shane Conner",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/shaneconner/canon-atlas.git"
+  },
+  "bugs": "https://github.com/shaneconner/canon-atlas/issues",
+  "homepage": "https://github.com/shaneconner/canon-atlas#readme"
+}

package/src/build.mjs

@@ -0,0 +1,37 @@
+/* Build: corpus in, one self-contained chart out. Or, with no corpus, the
+   pure-client app: one self-contained page that opens folders in the browser. */
+
+import { writeFileSync } from "node:fs";
+import { resolve } from "node:path";
+import { loadConfig, loadVectors } from "./config.mjs";
+import { scanCorpus } from "./scan.mjs";
+import { buildGraph } from "./graph.mjs";
+import { buildData, renderPage, renderAppPage } from "./page.mjs";
+
+export function build(rootArg, outArg) {
+  const root = resolve(rootArg || ".");
+  const out = resolve(outArg || "atlas.html");
+  const config = loadConfig(root);
+  config.vectors = loadVectors(root, config);
+  const docs = scanCorpus(root, config);
+  if (!docs.length) throw new Error(`no markdown documents under ${root}`);
+  const graph = buildGraph(docs);
+  const data = buildData(config, graph, "chart");
+  writeFileSync(out, renderPage(data));
+  return {
+    out,
+    documents: docs.length,
+    references: graph.edges.length,
+    clusters: data.clusters.length,
+    basis: data.basis,
+  };
+}
+
+/* The pure-client app carries no corpus: it reads folders in the browser. One
+   self-contained file, opened once, serves every project. */
+export function buildApp(outArg) {
+  const out = resolve(outArg || "atlas-app.html");
+  writeFileSync(out, renderAppPage());
+  return { out };
+}
+

package/src/cli.mjs

@@ -0,0 +1,120 @@
+#!/usr/bin/env node
+/* canon-atlas: an atlas over a catalog of markdown that references itself.
+
+   canon-atlas build [root] [-o atlas.html]   one self-contained chart
+   canon-atlas serve [root] [-p 4747]         the live atlas, with editing */
+
+import { spawn } from "node:child_process";
+import { build, buildApp } from "./build.mjs";
+import { embed } from "./embed.mjs";
+import { serve, serveApp } from "./serve.mjs";
+
+const USAGE = `canon-atlas: view a catalog of markdown documents as a constellation
+
+usage:
+  canon-atlas open [-p 4700]                 open the atlas in your browser,
+                                             starting the local host if needed;
+                                             made for a launcher entry, and it
+                                             exits after a day with no atlas tab
+  canon-atlas app [-o atlas-app.html]        write the pure-client app as one
+                                             file, to host anywhere static
+  canon-atlas embed [root] [-m PROVIDER]     write embeddings.json at the corpus
+                                             root, upgrading cluster color from
+                                             link structure to meaning; reruns
+                                             embed only what changed
+  canon-atlas build [root] [-o atlas.html]   write one self-contained chart
+  canon-atlas serve [root] [-p 4747]         serve one corpus live, with editing
+
+The root is a directory of markdown documents, current directory by default.
+A canon-atlas.json at the root defines collections; without one, a root that
+holds articles/ and journal/ gets the pi-canon preset, and anything else is
+read as a single mutable collection. An embeddings.json of path to vector
+upgrades cluster color from link structure to meaning.
+
+embed is the one command that touches a network, and only the provider named:
+the default is ollama (local, so nothing leaves the machine; OLLAMA_HOST
+overrides the address, model nomic-embed-text unless -m ollama:MODEL names
+another). -m openai[:MODEL] uses the OpenAI API instead, with OPENAI_API_KEY
+and optionally OPENAI_BASE_URL from the environment. A corpus embedded before
+keeps its recorded model on rerun; -m switches it, re-embedding everything.
+
+The app needs no root: it reads folders in the browser, so one page serves
+every project and each browser tab holds one. Anywhere the File System Access
+API runs (Chrome or Edge over http or https) it edits in place; elsewhere,
+and on file:// pages, it browses read-only, which is why hosting it (-p, or
+any static host) is the full experience.`;
+
+function parse(argv) {
+  const args = { _: [] };
+  for (let i = 0; i < argv.length; i++) {
+    const a = argv[i];
+    if (a === "-o" || a === "--out") args.out = argv[++i];
+    else if (a === "-p" || a === "--port") args.port = Number(argv[++i]);
+    else if (a === "-m" || a === "--model") args.model = argv[++i];
+    else if (a === "-h" || a === "--help") args.help = true;
+    else args._.push(a);
+  }
+  return args;
+}
+
+const args = parse(process.argv.slice(2));
+const cmd = args._[0];
+
+try {
+  if (args.help || !cmd) {
+    console.log(USAGE);
+  } else if (cmd === "open") {
+    const port = args.port ?? 4700;
+    const url = `http://127.0.0.1:${port}/`;
+    // Constants over knobs: a full day of no atlas tab, then the host exits.
+    const IDLE_MS = 24 * 60 * 60 * 1000;
+    const browse = () => {
+      const child = spawn("xdg-open", [url], { detached: true, stdio: "ignore" });
+      child.on("error", () => console.error(`xdg-open is missing; open ${url} yourself`));
+      child.unref();
+    };
+    try {
+      await serveApp(port, { idleMs: IDLE_MS, onIdle: () => process.exit(0) });
+      browse();
+      console.log(`${url}\nexits after a day with no atlas tab open`);
+    } catch (e) {
+      if (e && e.code === "EADDRINUSE") {
+        // A host is already on the port: reuse it if it is ours.
+        const ours = await fetch(url + "ping")
+          .then((r) => r.headers.get("x-canon-atlas") === "app")
+          .catch(() => false);
+        if (!ours) throw new Error(`port ${port} is taken by something else`);
+        browse();
+        // The running host serves the tab; this process is done. An explicit
+        // exit, because the probe fetch can hold the loop open on keep-alive.
+        setTimeout(() => process.exit(0), 300);
+      } else throw e;
+    }
+  } else if (cmd === "app") {
+    const r = buildApp(args.out);
+    console.log(`${r.out}\nopen it in a Chromium browser, then pick a folder`);
+  } else if (cmd === "embed") {
+    const r = await embed(args._[1], args.model, (p) => process.stderr.write(`\rembedding ${p}   `));
+    if (r.embedded) process.stderr.write("\n");
+    console.log(
+      `${r.file}\n${r.embedded} embedded, ${r.reused} unchanged` +
+        (r.removed ? `, ${r.removed} removed` : "") +
+        `, model ${r.model}\nbuild, serve, and the app pick it up on the next load`
+    );
+  } else if (cmd === "build") {
+    const r = build(args._[1], args.out);
+    console.log(
+      `${r.out}\n${r.documents} documents, ${r.references} references, ` +
+        `${r.clusters} clusters from ${r.basis === "embeddings" ? "embeddings" : "link structure"}`
+    );
+  } else if (cmd === "serve") {
+    const r = await serve(args._[1], args.port);
+    console.log(`canon-atlas over ${r.root}\nhttp://127.0.0.1:${r.port}/`);
+  } else {
+    console.error(`unknown command: ${cmd}\n\n${USAGE}`);
+    process.exit(2);
+  }
+} catch (e) {
+  console.error(String(e.message || e));
+  process.exit(1);
+}

package/src/clusters.mjs

@@ -0,0 +1,11 @@
+/* Semantic clusters: the color and region signal for the constellation.
+   Spherical k-means over embeddings when the corpus ships them, deterministic
+   greedy modularity over the link graph otherwise. The logic lives in
+   ui/pipeline.cjs so the browser clusters identically; this re-exports it. */
+
+import { createRequire } from "node:module";
+
+const P = createRequire(import.meta.url)("./ui/pipeline.cjs");
+
+export const communities = P.communities;
+export const assignClusters = P.assignClusters;

package/src/config.mjs

@@ -0,0 +1,42 @@
+/* Corpus configuration in Node: pick the raw config from a canon-atlas.json at
+   the root or from the directory shape, then hand off to the shared pipeline
+   for normalization. The pure parts (buildConfig, the presets, parseVectors,
+   collectionFor) live in ui/pipeline.cjs, so the browser builds the same config
+   from the same rules. */
+
+import { existsSync, readFileSync, statSync } from "node:fs";
+import { join } from "node:path";
+import { createRequire } from "node:module";
+
+const P = createRequire(import.meta.url)("./ui/pipeline.cjs");
+
+export const CONFIG_NAME = P.CONFIG_NAME;
+export const collectionFor = P.collectionFor;
+
+function isDir(p) {
+  try {
+    return statSync(p).isDirectory();
+  } catch {
+    return false;
+  }
+}
+
+/* Resolution order: an explicit canon-atlas.json at the root wins, then preset
+   detection (a store with both articles/ and journal/ is pi-canon), then the
+   bare default of one mutable collection over every markdown file. A config
+   that names a title but no collections keeps the detected preset. */
+export function loadConfig(root) {
+  const file = join(root, CONFIG_NAME);
+  const raw = existsSync(file) ? JSON.parse(readFileSync(file, "utf8")) : null;
+  const detect = () =>
+    isDir(join(root, "articles")) && isDir(join(root, "journal")) ? P.PI_CANON_PRESET : P.DEFAULT_CONFIG;
+  return P.buildConfig(P.composeConfig(raw, detect));
+}
+
+/* Optional embedding vectors at the path config.embeddings names. Absent is
+   normal; clustering falls back to link structure. */
+export function loadVectors(root, config) {
+  const file = join(root, config.embeddings);
+  if (!existsSync(file)) return null;
+  return P.parseVectors(JSON.parse(readFileSync(file, "utf8")));
+}

package/src/embed.mjs

@@ -0,0 +1,179 @@
+/* Embedding generation: the one network-touching command in the atlas.
+   Everything else is deterministic and offline; this writes the embeddings.json
+   that build, serve, and the app consume, and it only runs when asked.
+
+   The default provider is a local Ollama, so by default nothing leaves the
+   machine; OpenAI is the hosted alternative. Vectors are keyed by root-relative
+   path. A __meta__ entry records the model and a content hash per document
+   (the reader's parseVectors skips it, since its value is not a numeric array),
+   so a rerun embeds only what changed and a model switch re-embeds everything.
+   Writes are atomic and land after every batch, so an interrupted run keeps
+   its progress and resumes where it stopped. */
+
+import { createHash } from "node:crypto";
+import { existsSync, readFileSync, renameSync, writeFileSync } from "node:fs";
+import { request as httpRequest } from "node:http";
+import { request as httpsRequest } from "node:https";
+import { join, resolve } from "node:path";
+import { loadConfig } from "./config.mjs";
+import { scanCorpus } from "./scan.mjs";
+
+const BATCH = 16;
+const MAX_CHARS = 8000;
+
+const PROVIDERS = {
+  ollama: {
+    model: "nomic-embed-text",
+    url: () => (process.env.OLLAMA_HOST || "http://127.0.0.1:11434") + "/api/embed",
+    headers: () => ({ "Content-Type": "application/json" }),
+    body: (model, texts) => ({ model, input: texts }),
+    vectors: (json) => json.embeddings,
+    hint: "is Ollama running? set OLLAMA_HOST, or use -m openai:text-embedding-3-small",
+  },
+  openai: {
+    model: "text-embedding-3-small",
+    url: () => (process.env.OPENAI_BASE_URL || "https://api.openai.com") + "/v1/embeddings",
+    headers: () => {
+      if (!process.env.OPENAI_API_KEY) throw new Error("OPENAI_API_KEY is not set");
+      return { "Content-Type": "application/json", Authorization: `Bearer ${process.env.OPENAI_API_KEY}` };
+    },
+    body: (model, texts) => ({ model, input: texts }),
+    vectors: (json) => json.data.map((d) => d.embedding),
+    hint: "check OPENAI_API_KEY and OPENAI_BASE_URL",
+  },
+};
+
+function sha(text) {
+  return createHash("sha256").update(text).digest("hex");
+}
+
+/* Six decimals is well past what cosine distance can feel, and it keeps the
+   JSON a third the size a full double would write at high dimensions. */
+function round6(v) {
+  return Math.round(v * 1e6) / 1e6;
+}
+
+function atomicWrite(file, content) {
+  const tmp = file + ".atlas-tmp-" + process.pid;
+  writeFileSync(tmp, content);
+  renameSync(tmp, file);
+}
+
+/* Node's fetch gives up on any response whose headers take five minutes, and
+   a batch on a big local model legitimately thinks longer than that. Plain
+   node:http imposes no deadline: the model answers when it answers. */
+function post(url, headers, body) {
+  return new Promise((fulfil, reject) => {
+    const u = new URL(url);
+    const req = (u.protocol === "https:" ? httpsRequest : httpRequest)(
+      u,
+      { method: "POST", headers },
+      (res) => {
+        let text = "";
+        res.setEncoding("utf8");
+        res.on("data", (c) => (text += c));
+        res.on("end", () => fulfil({ status: res.statusCode, text }));
+      }
+    );
+    req.on("error", reject);
+    req.end(body);
+  });
+}
+
+/* modelArg is provider or provider:model, e.g. "ollama", "openai:text-embedding-3-large".
+   Without one, a corpus that was embedded before keeps its recorded model, so a
+   bare rerun never silently switches models and re-embeds the world. */
+export async function embed(rootArg, modelArg, log = () => {}) {
+  const root = resolve(rootArg || ".");
+  const config = loadConfig(root);
+
+  const file = join(root, config.embeddings);
+  let store = {};
+  if (existsSync(file)) {
+    try {
+      store = JSON.parse(readFileSync(file, "utf8"));
+    } catch {
+      store = {};
+    }
+  }
+
+  const spec = modelArg || (store.__meta__ && store.__meta__.model) || "ollama";
+  const sep = spec.indexOf(":");
+  const providerName = sep < 0 ? spec : spec.slice(0, sep);
+  const provider = PROVIDERS[providerName];
+  if (!provider) {
+    throw new Error(`unknown provider: ${providerName}; use ollama[:model] or openai[:model]`);
+  }
+  const model = sep < 0 ? provider.model : spec.slice(sep + 1);
+  const modelId = providerName + ":" + model;
+
+  const docs = scanCorpus(root, config);
+  if (!docs.length) throw new Error(`no markdown documents under ${root}`);
+
+  const meta =
+    store.__meta__ && store.__meta__.model === modelId
+      ? store.__meta__
+      : { model: modelId, hashes: {} };
+  store.__meta__ = meta;
+
+  // What needs embedding: new documents, changed documents, or everything
+  // after a model switch. Unchanged documents keep their stored vectors.
+  const current = Object.create(null);
+  const pending = [];
+  for (const d of docs) {
+    const text = (d.title + "\n\n" + d.body).slice(0, MAX_CHARS);
+    const hash = sha(text);
+    current[d.path] = true;
+    if (meta.hashes[d.path] === hash && Array.isArray(store[d.path])) continue;
+    pending.push({ path: d.path, text, hash });
+  }
+
+  // Vectors for documents that no longer exist come out.
+  let removed = 0;
+  for (const key of Object.keys(store)) {
+    if (key !== "__meta__" && !current[key]) {
+      delete store[key];
+      delete meta.hashes[key];
+      removed++;
+    }
+  }
+
+  let done = 0;
+  for (let i = 0; i < pending.length; i += BATCH) {
+    const batch = pending.slice(i, i + BATCH);
+    let res;
+    try {
+      res = await post(
+        provider.url(),
+        provider.headers(),
+        JSON.stringify(provider.body(model, batch.map((b) => b.text)))
+      );
+    } catch (e) {
+      throw new Error(`${providerName} unreachable at ${provider.url()} (${e.message}); ${provider.hint}`);
+    }
+    if (res.status < 200 || res.status >= 300) {
+      throw new Error(`${providerName} answered ${res.status}: ${res.text.slice(0, 300)}`);
+    }
+    const vecs = provider.vectors(JSON.parse(res.text));
+    if (!Array.isArray(vecs) || vecs.length !== batch.length) {
+      throw new Error(`${providerName} returned ${vecs ? vecs.length : "no"} vectors for ${batch.length} documents`);
+    }
+    batch.forEach((b, j) => {
+      store[b.path] = vecs[j].map(round6);
+      meta.hashes[b.path] = b.hash;
+    });
+    atomicWrite(file, JSON.stringify(store));
+    done += batch.length;
+    log(`${done}/${pending.length}`);
+  }
+  if (!pending.length && removed) atomicWrite(file, JSON.stringify(store));
+
+  return {
+    file,
+    model: modelId,
+    documents: docs.length,
+    embedded: pending.length,
+    reused: docs.length - pending.length,
+    removed,
+  };
+}

package/src/graph.mjs

@@ -0,0 +1,11 @@
+/* Link graph: the shared pipeline resolves every authored link through the same
+   resolver the reader uses, keeps the ones that land, and surfaces the ones
+   that do not as phantom nodes. This wrapper re-exports it for the Node build
+   path and the gate suite; the logic lives in ui/pipeline.cjs. */
+
+import { createRequire } from "node:module";
+
+const P = createRequire(import.meta.url)("./ui/pipeline.cjs");
+
+export const buildGraph = P.buildGraph;
+export const pagerank = P.pagerank;

package/src/markdown.mjs

@@ -0,0 +1,10 @@
+/* The reader panel's small markdown renderer. Everything is escaped first, so a
+   document can never inject markup; wikilinks and relative markdown links become
+   internal anchors the app wires to the graph. The logic lives in
+   ui/pipeline.cjs so the browser renders identically; this re-exports it. */
+
+import { createRequire } from "node:module";
+
+const P = createRequire(import.meta.url)("./ui/pipeline.cjs");
+
+export const renderMarkdown = P.renderMarkdown;