overleaf-forge 2.9.1 → 2.11.0

package/README.md

@@ -2,7 +2,7 @@
 
 [![npm](https://img.shields.io/npm/v/overleaf-forge.svg)](https://www.npmjs.com/package/overleaf-forge) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](#license) ![Node](https://img.shields.io/badge/node-%E2%89%A518-43853d.svg)
 
-A [Model Context Protocol](https://modelcontextprotocol.io) server that lets an AI assistant (Claude Code, Claude Desktop, or any MCP client) read, edit, compile, and verify an [Overleaf](https://www.overleaf.com) project over Overleaf's built-in **git** integration. The model edits a local clone with surgical, conflict-safe operations and pushes to Overleaf; nothing depends on scraping the web UI.
+A [Model Context Protocol](https://modelcontextprotocol.io) server that lets an AI assistant (Claude Code, Claude Desktop, or any MCP client) read, edit, compile, and verify an [Overleaf](https://www.overleaf.com) project over Overleaf's built-in **git** integration. The model edits a local clone with surgical, conflict-safe operations and publishes verified batches to Overleaf; nothing depends on scraping the web UI.
 
 > **Acknowledgement.** Forked from [mjyoo2/OverleafMCP](https://github.com/mjyoo2/OverleafMCP), the original Overleaf MCP server. This fork adds conflict-safe editing, binary/figure upload, a clean-build PASS/FAIL gate, citation and voice linting, snapshots, a bootstrap for recurring structured documents, per-project contexts, and a hardened git layer (no-shell `execFile`, credential-helper auth, error redaction).
 
@@ -49,7 +49,7 @@ For an iterative edit, build, and review loop on a large document the tool traff
 
 - Node.js ≥ 18 (ESM).
 - `git` on `PATH`.
-- A LaTeX distribution with `latexmk` (only for `compile_file` / `verify_build`; the rest works without it). The default engine is LuaLaTeX; `latexmk` is expected at `/Library/TeX/texbin` (MacTeX) or otherwise on `PATH`.
+- A LaTeX distribution with `latexmk` (only for `verify_build`; the rest works without it). The default engine is LuaLaTeX; `latexmk` is expected at `/Library/TeX/texbin` (MacTeX) or otherwise on `PATH`.
 - An Overleaf account with **Git integration** enabled (a paid feature at time of writing).
 
 ## Install
@@ -180,7 +180,7 @@ That creates `~/.overleaf-mcp/projects.json` from the example and copies the edi
 
 ### Where files live
 
-User state (the `projects.json`, per-project `contexts/`, customised `templates/`, and the git clones) lives in the **data home**, resolved as: `$OVERLEAF_MCP_HOME` if set, else the package directory when it already holds a `projects.json` (so an existing local clone keeps working untouched), else `~/.overleaf-mcp`. Bundled, read-only defaults (the templates and the stock writing-guidelines) ship inside the package; a copy you place in the data home overrides the bundled one.
+User state (the `projects.json`, per-project `contexts/`, customised `templates/`, and the git clones) lives in the **data home**, resolved as: `$OVERLEAF_MCP_HOME` if set, else the package directory when it already holds a `projects.json` (so an existing local clone keeps working untouched), else `~/.overleaf-mcp`. Bundled, read-only defaults (the templates and the stock writing-guidelines) ship inside the package; a copy you place in the data home overrides the bundled one. For personal writing rules, use `writing-guidelines.local.md` (gitignored, read first on every `get_context` call); it stays distinct from the bundled file even when the data home is the package directory.
 
 ### Getting Overleaf credentials
 
@@ -224,15 +224,17 @@ A typical editing session, in the model's words:
 4. *"Add a figure: upload `~/plots/fig1.png` to `figures/fig1.png`."* → `upload_file`
 5. *"Verify the build."* → `verify_build` → `✓ PASS — 12 pages`
 
-Every write commits and pushes to Overleaf; `verify_build` is the gate before calling the work done.
+Every write commits locally; `verify_build` is the gate before calling the work done, and `publish_changes` pushes the verified commits to Overleaf in one step. Set `settings.autoPush: true` (or pass `push: true`) to push on every write instead.
 
 ## Conflict safety
 
 Edits never silently overwrite a concurrent Overleaf change.
 
-- **`edit_file`** pulls the latest first (so a non-overlapping browser edit is absorbed), then replaces an exact anchor string. If the anchor is gone, the region changed since you read it, and the edit refuses rather than guessing. On the rare push race, git performs a real 3-way merge and the edit refuses only on a true overlap.
-- **`write_file`** (full-file create or overwrite) creates a new file freely. To overwrite an existing file it requires either the `baseSha` you got from `read_file` (a stale one is refused) or an explicit `overwrite: true`. It never merges a wholesale replacement; a push race refuses and resets.
+- **Local by default.** Write tools commit to the local clone and stay off the network unless `push: true` or `settings.autoPush` is set. `publish_changes` pushes the accumulated commits after verification. When any push is refused, only that operation's own commit is rolled back; earlier unpublished commits are never discarded.
+- **`edit_file`** replaces an exact anchor string. If the anchor is gone, the region changed since you read it, and the edit refuses rather than guessing. When pushing, it pulls first (so a non-overlapping browser edit is absorbed), and on the rare push race git performs a real 3-way merge and the edit refuses only on a true overlap.
+- **`write_file`** (full-file create or overwrite) creates a new file freely. To overwrite an existing file it requires either the `baseSha` you got from `read_file` (a stale one is refused) or an explicit `overwrite: true`. It never merges a wholesale replacement; a push race refuses and rolls back its own commit.
 - **`upload_file`** uses the same gate for binaries, never merges, and confines every destination path inside the project clone.
+- **`sync_project`** fetches and fast-forwards when the clone is only behind. On divergence it changes nothing and reports the local and remote commits; `strategy: "rebase"` replays local work onto Overleaf (aborting cleanly on conflict), and `strategy: "reset"` discards local work only with `confirm` equal to the reported head, after tagging `mcp-backup/*` copies of the old head and any uncommitted edits.
 - An explicit `projectName` that doesn't resolve is an **error**, never a silent fall-through to a different project, so a write cannot land in the wrong repo.
 
 ## Bootstrap for recurring structured documents
@@ -267,7 +269,7 @@ By the built-in convention this parses the name, locates the parent course folde
 | `read_file` | Read a file. The first line carries the file's `baseSha` (its git blob hash) for conflict-safe writes. |
 | `list_files` | List files in the project, filtered by extension. |
 | `get_sections` | List `\section` / `\subsection` / `\subsubsection` entries in a `.tex`. |
-| `get_section_content` | Pull a single section's body by title (level-aware: a section keeps its subsections). |
+| `get_section_content` | One section's body by exact, unique title, `\section` down to `\paragraph` (level-aware: a section keeps its subsections). `bundle: true` adds the referenced equation/figure blocks, bibliography entries and assets. |
 | `search_text` | Grep tracked files. Regex by default; `fixed` for a literal, `ignoreCase`, `extension` to scope. Returns `file:line:match`. |
 | `status_summary` | File count, main file, section count. |
 
@@ -275,7 +277,7 @@ By the built-in convention this parses the name, locates the parent course folde
 
 | Tool | Purpose |
 | --- | --- |
-| `edit_file` | Anchored `oldString` → `newString` edit + commit + push. Conflict-safe; auto-merges non-overlapping concurrent edits. Preferred for existing files. |
+| `edit_file` | Anchored `oldString` → `newString` edit + commit (push per `push` / `settings.autoPush`). Conflict-safe; auto-merges non-overlapping concurrent edits. Preferred for existing files. |
 | `write_file` | Create a new file, or overwrite one wholesale. Existing-file overwrite needs `baseSha` or `overwrite: true`. |
 | `upload_file` | Upload binary file(s) (figures) from a local path. Single or batch (one commit). Byte-exact, path-confined, same conflict gate. |
 
@@ -283,22 +285,23 @@ By the built-in convention this parses the name, locates the parent course folde
 
 | Tool | Purpose |
 | --- | --- |
-| `compile_file` | Compile with `latexmk` from the repo root, so the project `.latexmkrc`, reruns, and bibliography all apply; reports errors, undefined refs, overfull boxes. |
-| `verify_build` | Clean-from-scratch compile + PASS/FAIL verdict: PASS only with a PDF and zero errors / undefined references / undefined citations. Reports page count. The done-bar gate. |
+| `verify_build` | Compile with `latexmk` from the repo root (project `.latexmkrc`, reruns and bibliography apply) + PASS/FAIL verdict: PASS only with a PDF and zero errors / undefined references / undefined citations. Reports page count. Default is the clean from-scratch done-bar gate; `clean: false` is a quick incremental rebuild; `lint` adds voice-linter findings to the gate. |
+| `sync_project` | Fetch and reconcile with Overleaf: fast-forward, report divergence, or resolve it with `rebase` / confirmed `reset` (see Conflict safety). |
+| `publish_changes` | Verify the clean local HEAD and push every unpublished commit once. |
 
 **Citations, snapshots, voice**
 
 | Tool | Purpose |
 | --- | --- |
-| `add_citation` | Append a BibTeX entry to `refs.bib` (refuses a duplicate key) + push. |
+| `add_citation` | Append a BibTeX entry to `refs.bib` (refuses a duplicate key) + commit. |
 | `cite_lint` | Report undefined (`\cite` with no entry) and unused (entry never cited) citations. Read-only. |
 | `checkpoint` | Mark a local rollback point (a `mcp-snap/<label>` tag) before a risky edit. |
-| `restore` | Roll back to a checkpoint via a forward commit + push (no force, no history rewrite). |
-| `voice_lint` | Run a prose linter on a `.tex` (the bundled `examples/voice-lint.mjs` by default; override via `settings.voiceLinter`). Lints the local working copy as-is, never pulls. Read-only, advisory. |
+| `restore` | Roll back to a checkpoint via a forward commit (no force, no history rewrite). |
+| `voice_lint` | Run a prose linter on a `.tex` (the bundled `examples/voice-lint.mjs` by default; override via `settings.voiceLinter`). Lints the local working copy as-is, never pulls. Read-only and advisory; `verify_build` with `lint` makes it gating. |
 
 ## How it works
 
-Each project is a normal git clone of its Overleaf repo, kept under `repoDir` (or `localPath`). On every operation the server pulls the latest, performs the read/edit/build locally, and pushes. Git runs through `execFile` with argument arrays (no shell), so file paths, commit messages, and patterns can't inject commands. Authentication uses an inline git credential helper that reads the token from the process environment, so the token is never written into a remote URL, a command line, or an error message; the clone's `origin` stays token-free. `compile_file` and `verify_build` shell out to `latexmk` from the repo root so the project's own `.latexmkrc` governs the build.
+Each project is a normal git clone of its Overleaf repo, kept under `repoDir` (or `localPath`). `sync_project` is the only synchronization command. Read, search, lint, dependency, render and build tools use the existing clone without pulling. This gives a task a stable source snapshot and prevents a read from replacing local edits. Git runs through `execFile` with argument arrays (no shell), so file paths, commit messages, and patterns can't inject commands. Authentication uses an inline git credential helper that reads the token from the process environment, so the token is never written into a remote URL, a command line, or an error message; the clone's `origin` stays token-free.
 
 ## Testing & development
 
@@ -324,3 +327,17 @@ Forked from [mjyoo2/OverleafMCP](https://github.com/mjyoo2/OverleafMCP). The ori
 ## License
 
 MIT. See [LICENSE](LICENSE) if present, or treat this as MIT per the upstream project.
+
+## Efficient local reading and builds
+
+- `get_context({projectName, previousVersion})` returns a context version and omits unchanged content. The version covers project identity and the rendered guidance/context.
+- `get_section_content({projectName, filePath, sectionTitle, bundle: true, maxChars})` reads locally without pulling and returns directly referenced equation/figure blocks, matching BibTeX entries and asset paths. It reports missing matches and truncation. Macro-generated references, recursive TeX expansion and parenthesized BibTeX entries require a focused follow-up read.
+- All read, search, lint, dependency, render and build tools are local-only. Use `sync_project` explicitly before reading when needed. It fast-forwards only; divergence is reported, never auto-resolved.
+- Build output is compact by default. `verbose: true` includes a bounded log tail; the full log remains at the returned path.
+- `verify_build` can reuse a successful verification within the running server when project files, recorder inputs, tool binaries, environment and output artifacts are unchanged. `force: true` rebuilds. Missing recorder data, symlinks or executable build configuration conservatively disable reuse. The cache is in-memory and disappears on server restart. Custom build commands can have undeclared external dependencies, so projects with latexmkrc files or detected shell/Lua generation rebuild.
+- `controlled: true` runs `latexmk -norc -no-shell-escape` and permits caching in projects with a `.latexmkrc`, because the rc file cannot run. The mode still rejects shell escape, `minted` and Lua file generation. Add required absolute regular `externalInputs` so their content enters the cache key.
+- `dependency_index` records static TeX includes, figures, labels, references, citations and declared values. `change_report` compares a later index to a retained baseline and names affected sections. Dynamic macros are reported as unresolved rather than guessed.
+- `render_pages` caches a requested PDF page by PDF content hash, page, DPI and renderer version. `usage_stats` exposes aggregate in-process timing, response-size and cache-hit measurements without retaining request or document text.
+- `apply_changes` verifies an SHA-guarded UTF-8 multi-file candidate in an isolated worktree and fast-forwards one local commit only if verification passes. `publish_changes` separately re-verifies its exact clean revision, then pushes it once, carrying every unpublished local commit. Neither operation pulls, resets, retries or silently merges.
+
+After updating the server, reconnect the MCP client so it reloads the process and tool schemas. Package publication is separate from installing these local changes.

package/dependency-index.js

@@ -0,0 +1,189 @@
+import { createHash } from 'node:crypto';
+import { lstat, readFile, realpath, readdir } from 'node:fs/promises';
+import path from 'node:path';
+
+const parseCache = new Map();
+const snapshots = new Map();
+const MAX_FILES = 1000;
+const MAX_RESPONSE = 5000;
+const MAX_SNAPSHOTS = 8;
+
+const digest = bytes => createHash('sha256').update(bytes).digest('hex');
+const lineAt = (text, offset) => text.slice(0, offset).split('\n').length;
+const rel = (root, file) => path.relative(root, file).split(path.sep).join('/');
+
+function inside(root, candidate) {
+  const r = path.resolve(root);
+  const c = path.resolve(candidate);
+  return c === r || c.startsWith(`${r}${path.sep}`);
+}
+
+function withoutComments(source) {
+  return source.replace(/(^|[^\\])%[^\n]*/g, '$1');
+}
+
+function sectionFor(sections, offset) {
+  let result = null;
+  for (const section of sections) {
+    if (section.offset <= offset) result = section;
+    else break;
+  }
+  return result;
+}
+
+function parseFile(file, source, hash) {
+  const text = withoutComments(source);
+  const sections = [];
+  const sectionRe = /\\(part|chapter|section|subsection|subsubsection|paragraph|subparagraph)\*?\{((?:[^{}]|\{[^{}]*\})*)\}/g;
+  let match;
+  while ((match = sectionRe.exec(text))) sections.push({
+    id: `${file}#${match[2]}`,
+    title: match[2], type: match[1], line: lineAt(text, match.index), offset: match.index,
+  });
+  for (let i = 0; i < sections.length; i++) {
+    const nextOffset = i + 1 < sections.length ? sections[i + 1].offset : source.length;
+    sections[i].endLine = i + 1 < sections.length ? sections[i + 1].line - 1 : source.split('\n').length;
+    // Hash the section's own source span. This keeps an unchanged subsection
+    // stable when text in a sibling section shifts its line number.
+    sections[i].hash = digest(source.slice(sections[i].offset, nextOffset));
+    if (i > 0 && sections[i - 1].id === sections[i].id) sections[i].id += `:${i + 1}`;
+  }
+  const declarations = [];
+  const labels = [];
+  const refs = [];
+  const citations = [];
+  const includes = [];
+  const graphics = [];
+  const values = [];
+  if (/\.bib$/i.test(file)) {
+    const bibRe = /@\w+\s*\{\s*([^,\s]+)\s*,/g;
+    while ((match = bibRe.exec(text))) citations.push({ kind: 'bib', symbol: match[1], line: lineAt(text, match.index), section: null });
+  }
+  const declarationRe = /\\(?:newcommand|renewcommand|providecommand)\s*\{\\([^}]+)\}(?:\s*\[[^\]]*\])?\s*\{([^{}]*)\}/g;
+  while ((match = declarationRe.exec(text))) {
+    const symbol = `\\${match[1]}`;
+    const declaration = { symbol, value: match[2], line: lineAt(text, match.index), section: sectionFor(sections, match.index)?.id ?? null };
+    declarations.push(declaration); values.push(declaration);
+  }
+  const labelRe = /\\label\s*\{([^}]+)\}/g;
+  while ((match = labelRe.exec(text))) labels.push({ symbol: match[1], line: lineAt(text, match.index), section: sectionFor(sections, match.index)?.id ?? null });
+  const refRe = /\\(ref|pageref|autoref|cref|Cref)\s*\{([^}]+)\}/g;
+  while ((match = refRe.exec(text))) refs.push({ kind: match[1], symbol: match[2], line: lineAt(text, match.index), section: sectionFor(sections, match.index)?.id ?? null, dynamic: /\\|\$|#/.test(match[2]) });
+  const citeRe = /\\cite[a-zA-Z*]*\s*(?:\[[^\]]*\]\s*)?\{([^}]+)\}/g;
+  while ((match = citeRe.exec(text))) for (const symbol of match[1].split(',').map(x => x.trim()).filter(Boolean)) citations.push({ kind: 'cite', symbol, line: lineAt(text, match.index), section: sectionFor(sections, match.index)?.id ?? null });
+  const includeRe = /\\(input|include|includegraphics)\s*(?:\[[^\]]*\])?\s*\{([^}]+)\}/g;
+  while ((match = includeRe.exec(text))) {
+    const item = { kind: match[1], target: match[2].trim(), line: lineAt(text, match.index), section: sectionFor(sections, match.index)?.id ?? null, dynamic: /\\|\$|#/.test(match[2]) };
+    (match[1] === 'includegraphics' ? graphics : includes).push(item);
+  }
+  return { file, hash, sections, declarations, labels, refs, citations, includes, graphics, values };
+}
+
+async function filesUnder(root, notices) {
+  const result = [];
+  async function walk(dir) {
+    let entries;
+    try { entries = await readdir(dir, { withFileTypes: true }); } catch (error) { notices.push(`cannot read ${rel(root, dir)}: ${error.message}`); return; }
+    for (const entry of entries) {
+      const full = path.join(dir, entry.name);
+      if (entry.isSymbolicLink()) { notices.push(`skipped symlink ${rel(root, full)}`); continue; }
+      if (entry.isDirectory()) { await walk(full); continue; }
+      if (entry.isFile() && /\.(?:tex|sty|cls|bib|bbx|cbx|ltx)$/i.test(entry.name)) result.push(full);
+    }
+  }
+  await walk(root);
+  return result.sort();
+}
+
+function resolveTarget(target, from, root, files) {
+  if (!target || /[\\$#]/.test(target)) return null;
+  const base = path.resolve(root, path.dirname(from), target);
+  const candidates = [base, ...(!path.extname(base) ? ['.tex', '.sty', '.cls', '.bib'].map(ext => `${base}${ext}`) : [])];
+  return candidates.find(candidate => files.has(rel(root, candidate))) ? rel(root, candidates.find(candidate => files.has(rel(root, candidate)))) : null;
+}
+
+function cap(list, limit, field, truncated) {
+  if (list.length <= limit) return list;
+  truncated[field] = true;
+  return list.slice(0, limit);
+}
+
+export async function dependencyIndex(root, { changedFiles = [], changedSymbols = [] } = {}) {
+  const rootReal = await realpath(root);
+  const notices = [];
+  const paths = await filesUnder(rootReal, notices);
+  const truncated = {};
+  const files = new Map();
+  const pathSet = new Set(paths.map(file => rel(rootReal, file)));
+  for (const file of paths.slice(0, MAX_FILES)) {
+    const relative = rel(rootReal, file);
+    const bytes = await readFile(file);
+    const hash = digest(bytes);
+    const key = `${rootReal}:${relative}`;
+    const old = parseCache.get(key);
+    const parsed = old?.hash === hash ? old : parseFile(relative, bytes.toString('utf8'), hash);
+    parseCache.set(key, parsed);
+    files.set(relative, parsed);
+  }
+  if (paths.length > MAX_FILES) truncated.files = true;
+  const labels = new Map();
+  const citations = new Map();
+  for (const file of files.values()) {
+    for (const label of file.labels) labels.set(label.symbol, { file: file.file, section: label.section, line: label.line });
+    for (const citation of file.citations) citations.set(citation.symbol, { file: file.file, section: citation.section, line: citation.line });
+  }
+  const edges = [];
+  const unresolved = [];
+  for (const file of files.values()) {
+    const add = (kind, item, target, symbol = null) => edges.push({ kind, from: { file: file.file, section: item.section, line: item.line }, to: target ? { file: target.file, section: target.section ?? null, line: target.line ?? null, symbol } : { symbol: target ?? symbol, unresolved: true } });
+    for (const item of file.refs) { const target = labels.get(item.symbol); add(item.kind, item, target, item.symbol); if (!target) unresolved.push({ kind: item.kind, file: file.file, section: item.section, line: item.line, symbol: item.symbol, dynamic: item.dynamic }); }
+    for (const item of file.citations) { const target = citations.get(item.symbol); add('cite', item, target, item.symbol); if (!target) unresolved.push({ kind: 'cite', file: file.file, section: item.section, line: item.line, symbol: item.symbol }); }
+    for (const item of [...file.includes, ...file.graphics]) { const targetFile = resolveTarget(item.target, file.file, rootReal, pathSet); add(item.kind, item, targetFile ? { file: targetFile } : item.target); if (!targetFile) unresolved.push({ kind: item.kind, file: file.file, section: item.section, line: item.line, symbol: item.target, dynamic: item.dynamic }); }
+  }
+  const changed = new Set();
+  for (const file of changedFiles) {
+    const candidate = path.resolve(rootReal, file);
+    if (!inside(rootReal, candidate)) throw new Error(`changed file must be inside root: ${file}`);
+    changed.add(rel(rootReal, candidate));
+  }
+  const affected = edges.filter(edge => changed.has(edge.to.file) || (edge.to.symbol && changedSymbols.includes(edge.to.symbol))).map(edge => edge.from);
+  const version = digest(JSON.stringify([...files].map(([file, value]) => [file, value.hash])));
+  return { version, files: cap([...files.values()].map(({ file, hash, sections, declarations }) => ({ file, hash, sections, symbols: declarations.map(item => item.symbol), definitions: declarations.slice(0, 200) })), MAX_FILES, 'files', truncated), edges: cap(edges, MAX_RESPONSE, 'edges', truncated), unresolved: cap(unresolved, MAX_RESPONSE, 'unresolved', truncated), affectedSections: cap([...new Map(affected.map(x => [x.file + '#' + x.section + ':' + x.line, x])).values()], MAX_RESPONSE, 'affectedSections', truncated), notices, truncated, truncatedFields: Object.keys(truncated) };
+}
+
+export async function changeReport(root, baselineVersion) {
+  const index = await dependencyIndex(root);
+  const key = await realpath(root);
+  const previous = baselineVersion ? snapshots.get(key)?.get(baselineVersion) : null;
+  const firstCall = !baselineVersion;
+  const baselineFound = firstCall || Boolean(previous);
+  const oldFiles = previous?.files ?? [];
+  const oldMap = new Map(oldFiles.map(file => [file.file, file]));
+  const currentMap = new Map(index.files.map(file => [file.file, file]));
+  const allFileNames = new Set([...oldMap.keys(), ...currentMap.keys()]);
+  const changedFiles = [...allFileNames].filter(file => oldMap.get(file)?.hash !== currentMap.get(file)?.hash).sort();
+  const removedFiles = [...oldMap.keys()].filter(file => !currentMap.has(file)).sort();
+  const changedFileSet = new Set(changedFiles);
+  const oldEdges = previous?.edges ?? [];
+  const affectedEdges = firstCall ? [] : [...index.edges, ...oldEdges].filter(edge => changedFileSet.has(edge.to.file));
+  const affectedSections = [...new Map(affectedEdges.map(edge => [edge.from.file + '#' + edge.from.section + ':' + edge.from.line, edge.from])).values()];
+  const notices = [...index.notices];
+  if (baselineVersion && !previous) notices.push(`baseline version not found: ${baselineVersion}; full refresh required`);
+  const changedFileHashes = Object.fromEntries(index.files.filter(file => changedFiles.includes(file.file)).map(file => [file.file, file.hash]));
+  const changedSections = [];
+  if (!firstCall && previous) {
+    for (const file of allFileNames) {
+      const before = new Map((oldMap.get(file)?.sections ?? []).map(section => [section.id, section]));
+      const after = new Map((currentMap.get(file)?.sections ?? []).map(section => [section.id, section]));
+      for (const id of new Set([...before.keys(), ...after.keys()])) {
+        if (before.get(id)?.hash !== after.get(id)?.hash) changedSections.push({ file, section: id, beforeHash: before.get(id)?.hash ?? null, hash: after.get(id)?.hash ?? null, removed: !after.has(id) });
+      }
+    }
+  }
+  const report = { version: index.version, baselineVersion: baselineVersion ?? null, baselineFound, fullRefreshRequired: Boolean(baselineVersion && !previous), firstCall, changedFiles: firstCall || !previous ? [] : changedFiles, removedFiles: firstCall || !previous ? [] : removedFiles, changedFileHashes, changedSections: cap(changedSections, MAX_RESPONSE, 'changedSections', index.truncated), affectedSections: cap(affectedSections, MAX_RESPONSE, 'affectedSections', index.truncated), affectedRefs: cap(affectedEdges, MAX_RESPONSE, 'affectedRefs', index.truncated), suggestions: firstCall || !previous ? [] : affectedSections.map(section => ({ file: section.file, section: section.section, action: 'review references and recompile' })), unresolved: index.unresolved, notices, truncated: index.truncated, truncatedFields: Object.keys(index.truncated) };
+  if (!snapshots.has(key)) snapshots.set(key, new Map());
+  snapshots.get(key).set(index.version, index);
+  const retained = snapshots.get(key);
+  while (retained.size > MAX_SNAPSHOTS) retained.delete(retained.keys().next().value);
+  return report;
+}

package/efficiency.js

@@ -0,0 +1,115 @@
+import { createHash } from 'node:crypto';
+import { lstat, readFile, readdir, realpath } from 'node:fs/promises';
+import path from 'node:path';
+
+const hash = value => createHash('sha256').update(value).digest('hex');
+export function versionedContext(key, text, previousVersion) {
+  const version = hash(JSON.stringify([key, text]));
+  return { version, unchanged: previousVersion === version, text: previousVersion === version ? 'Context unchanged.' : text };
+}
+
+// Cache keys include file contents, not mtimes. Hash external recorder inputs too;
+// an edited image, bibliography, package or deleted PDF must invalidate reuse.
+// Controlled mode is safe to cache only when the caller also controls the
+// latexmk invocation. It relaxes the project-config check, while executable
+// TeX features remain ineligible because their dependency closure is opaque.
+export function controlledBuildOptions(options = {}) {
+  if (!options || typeof options !== 'object' || Array.isArray(options)) throw new TypeError('build options must be an object');
+  const controlled = options.controlled === true;
+  const externalInputs = options.externalInputs ?? [];
+  if (!Array.isArray(externalInputs) || externalInputs.some(input => typeof input !== 'string' || !path.isAbsolute(input))) {
+    throw new TypeError('externalInputs must contain absolute paths');
+  }
+  return { sourcesOnly: options.sourcesOnly === true, controlled, externalInputs: [...new Set(externalInputs)] };
+}
+
+export async function buildFingerprint(root, entry, engine, options = {}) {
+  const { sourcesOnly, controlled, externalInputs } = controlledBuildOptions(options);
+  const records = [];
+  const job = path.resolve(root, entry.replace(/\.tex$/, ''));
+  const generated = new Set(['aux','log','pdf','fls','fdb_latexmk','synctex.gz','toc','out','bbl','bcf','blg','run.xml','lof','lot'].map(e => `${job}.${e}`));
+  let unsafe = false;
+  const visit = async dir => {
+    for (const d of (await readdir(dir, { withFileTypes: true })).sort((a,b)=>a.name.localeCompare(b.name))) {
+      if (d.name === '.git') continue;
+      const f = path.join(dir,d.name);
+      if (d.isSymbolicLink()) { unsafe = true; continue; }
+      if (d.isDirectory()) await visit(f);
+      else if (d.isFile() && !(sourcesOnly && generated.has(f))) {
+        const bytes = await readFile(f);
+        records.push([f,hash(bytes)]);
+        // Executable config and shell/Lua-driven generation can read dependencies
+        // outside the TeX recorder. Rebuild rather than certify an incomplete key.
+        if ((!controlled && /latexmkrc$/.test(f)) || (/\.(tex|sty|cls)$/.test(f) && /\\(?:write18|directlua|inputminted)|\\begin\{minted\}/.test(bytes.toString()))) unsafe = true;
+      }
+    }
+  };
+  await visit(root);
+  for (const input of externalInputs) {
+    const stat = await lstat(input).catch(() => null);
+    if (!stat || !stat.isFile() || stat.isSymbolicLink()) throw new Error(`external input must be a regular non-symlink file: ${input}`);
+    records.push([input, hash(await readFile(input))]);
+  }
+  if (unsafe && !sourcesOnly) return null;
+  if (!sourcesOnly) {
+    const fls = await readFile(`${job}.fls`,'utf8');
+    const inputs = new Set(fls.split(/\r?\n/).filter(l=>l.startsWith('INPUT ')).map(l=>path.resolve(root,l.slice(6))));
+    if (!inputs.size) return null;
+    for (const f of [...inputs].sort()) records.push([await realpath(f),hash(await readFile(f))]);
+    for (const tool of ['latexmk',engine]) {
+      const f = await realpath(`/Library/TeX/texbin/${tool}`);
+      records.push([f,hash(await readFile(f))]);
+    }
+  }
+  return hash(JSON.stringify([engine,entry,process.env,new Date().toDateString(),{ controlled, externalInputs },records]));
+}
+
+export function sectionText(content, title) {
+  const re = /\\(section|subsection|subsubsection|paragraph)\*?\{((?:[^{}]|\{[^{}]*\})*)\}/g;
+  const headings = [...content.matchAll(re)];
+  const matches = headings.filter(m=>m[2]===title);
+  if (matches.length !== 1) throw new Error(`Expected one section titled "${title}"; found ${matches.length}.`);
+  const ranks = {section:1,subsection:2,subsubsection:3,paragraph:4};
+  const t = matches[0];
+  const next = headings.find(m=>m.index>t.index && ranks[m[1]]<=ranks[t[1]]);
+  return content.slice(t.index,next?.index ?? content.length);
+}
+
+// Brace scanning preserves nested BibTeX values and equation/figure bodies.
+function entries(text) {
+  const out=[]; const re=/@\w+\s*\{\s*([^,\s]+)\s*,/g; let m;
+  while ((m=re.exec(text))) {
+    let depth=1,i=re.lastIndex;
+    for (;i<text.length && depth;i++) { if (text[i-1]==='\\') continue; if(text[i]==='{')depth++; if(text[i]==='}')depth--; }
+    out.push({key:m[1],text:text.slice(m.index,i)}); re.lastIndex=i;
+  }
+  return out;
+}
+export async function sectionBundle(root, filePath, title, maxChars = 16000) {
+  const full=path.resolve(root,filePath);
+  if (!full.startsWith(path.resolve(root)+path.sep)) throw new Error('Section path must be inside project');
+  if (!(await realpath(full)).startsWith(await realpath(root) + path.sep)) throw new Error('Section symlink escapes project');
+  const source=await readFile(full,'utf8'); const body=sectionText(source,title);
+  const wanted=new Set([...body.matchAll(/\\(?:auto|eq|page)?ref\{([^}]+)\}/g)].map(m=>m[1]));
+  const citations=new Set([...body.matchAll(/\\(?:cite\w*|autocite|parencite|textcite)\*?(?:\[[^\]]*\])*\{([^}]+)\}/g)].flatMap(m=>m[1].split(',').map(k=>k.trim())));
+  const blocks=[];const bibliography=[];
+  const walk=async dir=>{
+    for(const d of await readdir(dir,{withFileTypes:true})) {
+      if(d.name==='.git'||d.isSymbolicLink())continue;
+      const f=path.join(dir,d.name); if(d.isDirectory()){await walk(f);continue;}
+      if(!/\.(tex|bib)$/.test(f))continue;
+      const t=await readFile(f,'utf8');
+      if(f.endsWith('.bib')) { for(const e of entries(t))if(citations.has(e.key))bibliography.push({file:path.relative(root,f),...e}); continue; }
+      for(const m of t.matchAll(/\\begin\{(figure\*?|equation\*?|align\*?|gather\*?|multline\*?)\}[\s\S]*?\\end\{\1\}/g)) {
+        const labels=[...m[0].matchAll(/\\label\{([^}]+)\}/g)].map(x=>x[1]);
+        if(labels.some(l=>wanted.has(l))) blocks.push({file:path.relative(root,f),labels,text:m[0]});
+      }
+    }
+  }; await walk(root);
+  const resolved=new Set(blocks.flatMap(b=>b.labels));
+  const result={file:filePath,section:title,body,blocks,bibliography,unresolvedLabels:[...wanted].filter(l=>!resolved.has(l)),unresolvedCitations:[...citations].filter(k=>!bibliography.some(e=>e.key===k)),assets:[...body.matchAll(/\\(?:includegraphics(?:\[[^\]]*\])?|input)\{([^}]+)\}/g)].map(m=>m[1]),truncated:false};
+  // Never silently truncate: preserve the main section first and identify omissions.
+  while(JSON.stringify(result).length>maxChars && (result.blocks.length||result.bibliography.length)) {result.truncated=true;if(result.blocks.length)result.blocks.pop();else result.bibliography.pop();}
+  if(JSON.stringify(result).length>maxChars){result.truncated=true;result.body=body.slice(0,Math.max(0,maxChars-JSON.stringify({...result,body:''}).length-100));}
+  return result;
+}