agent-doc-system 1.4.0 → 1.5.1

package/CHANGELOG.md

@@ -9,6 +9,36 @@ refused by the next with `AGENTDOC_COMPILER_MISMATCH` until it is recompiled.
 
 ## [Unreleased]
 
+## [1.5.1] - 2026-11-13
+
+### Fixed
+
+- Evaluation and optimization entrypoints now execute when invoked through a symbolic link, including installed paths under macOS temporary-directory aliases. Previously they could exit successfully without producing output.
+- CI now exercises agent evaluation and package installation on macOS in addition to the Node 22/24/26 Linux matrix.
+
+Recompile committed graphs after upgrading, as the compiler version is now 1.5.1.
+
+## [1.5.0] - 2026-11-13
+
+### Added
+
+- Task-oriented `prepare`, `expand` and `refresh` JSON operations with versioned facts, evidence references, explicit coverage, snapshot-bound pagination and exact serialized token budgets for `o200k_base` and `cl100k_base`.
+- Local persistent compilation reuse, shared reads and content-addressed JSON parsing. Changed files, directory probes, ignore rules and compiler inputs invalidate reuse; full compilation remains the reference.
+- Session check-result imports with pinned execution evidence and conservative snapshot invalidation, plus an opt-in JavaScript TypeScript symbol-candidate index that never narrows checks.
+- An agent evaluation runner with independent validators, a 30-task synthetic starter corpus, a local Codex driver, bounded campaigns and conservative candidate selection. No model-efficiency improvement is claimed from the starter corpus.
+- Local context benchmarks with graph and packet equivalence checks.
+
+### Fixed
+
+- Default Markdown now reports capped conflicts and checks, repository gates, typed relationships and runtime facts. Settled conflicts are distinguished from unresolved ones.
+- Test fixtures no longer inherit local Git commit or tag signing requirements.
+
+### Upgrade
+
+- Recompile committed graphs with `agentdoc compile` after upgrading.
+- Exact token budgets use the pinned `js-tiktoken` runtime dependency. The compiler itself remains independent of model APIs. See [the task context protocol](docs/CONTEXT.md).
+
+
 ## [1.4.0] - 2026-11-10
 
 `impact` now treats a change under a Drizzle migration directory as a change
@@ -419,7 +449,9 @@ First public release.
   for project health.
 - An Agent Skill under `skills/agent-doc-system/`.
 
-[Unreleased]: https://github.com/lbframe/agent-doc-system/compare/v1.4.0...HEAD
+[Unreleased]: https://github.com/lbframe/agent-doc-system/compare/v1.5.1...HEAD
+[1.5.1]: https://github.com/lbframe/agent-doc-system/compare/v1.5.0...v1.5.1
+[1.5.0]: https://github.com/lbframe/agent-doc-system/compare/v1.4.0...v1.5.0
 [1.4.0]: https://github.com/lbframe/agent-doc-system/compare/v1.3.0...v1.4.0
 [1.3.0]: https://github.com/lbframe/agent-doc-system/compare/v1.2.0...v1.3.0
 [1.2.0]: https://github.com/lbframe/agent-doc-system/compare/v1.1.0...v1.2.0

package/INSTALL_FOR_AGENTS.md

@@ -37,7 +37,7 @@ npm install -g github:lbframe/agent-doc-system
 The command is idempotent: run it again to upgrade to the latest `main`.
 If a specific ref is required, append it: `github:lbframe/agent-doc-system#<ref>`.
 
-The package has zero runtime dependencies. If `npm install -g` fails on
+The task context commands use a pinned tokenizer dependency. If `npm install -g` fails on
 permissions, do **not** use `sudo`; fix the npm global prefix instead
 (`npm config set prefix ~/.npm-global` and add `~/.npm-global/bin` to `PATH`),
 or report the blocker.

package/README.md

@@ -45,6 +45,10 @@ AgentDoc gives them a map.
 
 ![AgentDoc in action — query, impact, check](docs/assets/readme/agentdoc-in-action.webp)
 
+## Task context for agents
+
+Use `agentdoc prepare` with a JSON task request to obtain a versioned context packet with an exact token budget. `expand` continues a partial packet; `refresh` updates retained context after edits. [Protocol and examples](docs/CONTEXT.md).
+
 ## Three commands, one workflow
 
 ```bash
@@ -102,7 +106,7 @@ Measured, reproducible, and with no model in the loop. Method and caveats are in
 - **Seconds to map a real repository.** A fresh `init` → `scaffold` → `compile` takes 1.0–4.6 s on four public repositories, from 141 to 5,142 files (axum, pydantic-ai, full-stack-fastapi-template, create-t3-turbo). No edit was needed beyond choosing each component's placement.
 - **A briefing, not a dump.** A `query` answer is about 100–260 tokens: what the change reaches, what to read, and which checks to run.
 - **The same graph every time.** Compiles are byte-identical, and CI can verify it.
-- **Small and strict.** Zero runtime dependencies, a package of about 285 kB packed, 21 adapters, and 709/709 cases on the official TOML 1.0 conformance suite.
+- **Small and strict.** A pinned tokenizer dependency, 21 adapters, and 709/709 cases on the official TOML 1.0 conformance suite.
 
 ## Install manually
 

package/bin/agentdoc.mjs

@@ -188,7 +188,16 @@ try {
     process.stdout.write(commandHelp(cmd));
     process.exit(0);
   }
-  if (cmd === "validate") {
+  if (["prepare", "expand", "refresh"].includes(cmd)) {
+    if (positional.length) throw new UsageError("context operations accept JSON via --request or stdin");
+    const file = opt("request");
+    const raw = fs.readFileSync(!file || file === "-" ? 0 : file, "utf8");
+    if (Buffer.byteLength(raw) > 1024 * 1024) throw new UsageError("context request exceeds 1 MiB");
+    const request = JSON.parse(raw);
+    const { prepare, expand, refresh, serializePacket } = await import("../core/context/index.mjs");
+    const result = ({ prepare, expand, refresh })[cmd](ROOT, request);
+    process.stdout.write(serializePacket(result));
+  } else if (cmd === "validate") {
     const res = compile(ROOT);
     printDiagnostics(res);
     if (res.errors.length) fail(res.errors, res.stats);
@@ -492,6 +501,12 @@ try {
     usage(2);
   }
 } catch (e) {
+  if (e instanceof UsageError || e instanceof SyntaxError && ["prepare", "expand", "refresh"].includes(cmd)) {
+    console.error("ERROR " + CODES.CONTEXT_REQUEST + ": " + e.message); process.exit(2);
+  }
+  if (Object.values(CODES).includes(e.code)) {
+    console.error("ERROR " + e.code + ": " + e.message); process.exit(1);
+  }
   if (e instanceof AgentDocError) {
     console.error("ERROR " + e.format());
     process.exit(1);

package/bin/commands.mjs

@@ -13,6 +13,21 @@ const JSON_FLAG = { json: { desc: "print the machine-readable report" } };
 const JSON_ALWAYS = { json: { desc: "accepted for explicitness; the output is always JSON" } };
 
 export const COMMANDS = {
+  prepare: {
+    synopsis: "agentdoc prepare [--request file|-]",
+    summary: "prepare a versioned task context packet from JSON (stdin by default)",
+    flags: { request: { value: "file|-", desc: "JSON request file, or - for stdin" }, ...JSON_ALWAYS },
+  },
+  expand: {
+    synopsis: "agentdoc expand [--request file|-]",
+    summary: "expand a packet continuation on the same snapshot",
+    flags: { request: { value: "file|-", desc: "JSON continuation request" }, ...JSON_ALWAYS },
+  },
+  refresh: {
+    synopsis: "agentdoc refresh [--request file|-]",
+    summary: "refresh context with explicit retained record IDs",
+    flags: { request: { value: "file|-", desc: "JSON refresh request" }, ...JSON_ALWAYS },
+  },
   init: {
     synopsis: "agentdoc init [--force]",
     summary: "write a configuration and documentation skeleton",

package/bin/usage.txt

@@ -4,6 +4,10 @@ agentdoc — portable documentation & software-catalog system
   agentdoc --help                    show this help
   agentdoc <command> --help          every flag one command accepts (also: agentdoc help <command>)
 
+  agentdoc prepare [--request file|-] task context packet (JSON stdin/stdout)
+  agentdoc expand [--request file|-]  continue a packet on its snapshot
+  agentdoc refresh [--request file|-] update retained context after changes
+
   agentdoc init [--force]            write a configuration and documentation skeleton
                                      (docs skeleton goes to the configured docs.* paths;
                                      agentdoc/docs/ when there is no configuration yet)

package/core/cache/compiled.mjs

@@ -0,0 +1,34 @@
+// One bounded, atomic cache record per repository. Cache errors only cost work.
+import fs from "node:fs";
+import path from "node:path";
+import { randomUUID } from "node:crypto";
+import { sha256Hex } from "../fsx.mjs";
+
+const LIMIT = 32 * 1024 * 1024;
+const digest = (record) => sha256Hex(JSON.stringify([record.key, record.serialized, record.stats]));
+export function readCompiledCache(file, key) {
+  try {
+    if (fs.statSync(file).size > LIMIT) return null;
+    const record = JSON.parse(fs.readFileSync(file, "utf8"));
+    if (record.version !== 1 || record.key !== key || typeof record.serialized !== "string") return null;
+    if (!record.stats || digest(record) !== record.digest) return null;
+    return record;
+  } catch { return null; }
+}
+
+export function writeCompiledCache(file, record) {
+  let temporary;
+  try {
+    for (const dir of [path.dirname(file), path.dirname(path.dirname(file))]) {
+      if (fs.existsSync(dir) && fs.lstatSync(dir).isSymbolicLink()) return false;
+    }
+    const text = JSON.stringify({ ...record, version: 1, digest: digest(record) });
+    if (Buffer.byteLength(text) > LIMIT) return false;
+    fs.mkdirSync(path.dirname(file), { recursive: true });
+    temporary = file + "." + randomUUID() + ".tmp";
+    fs.writeFileSync(temporary, text, { flag: "wx", mode: 0o600 });
+    fs.renameSync(temporary, file);
+    return true;
+  } catch { return false; }
+  finally { if (temporary) { try { fs.rmSync(temporary, { force: true }); } catch { /* cache is optional */ } } }
+}

package/core/cache/parses.mjs

@@ -0,0 +1,33 @@
+import { sha256Hex } from "../fsx.mjs";
+
+// Bounded content-addressed JSON parsing across snapshots in one process.
+// Values are cloned on both boundaries: extractors may mutate their result.
+export class ParseCache {
+  constructor(maxBytes = 8 * 1024 * 1024) {
+    this.maxBytes = maxBytes;
+    this.bytes = 0;
+    this.entries = new Map();
+  }
+  get(text) {
+    const key = sha256Hex("json@1\0" + text);
+    const entry = this.entries.get(key);
+    if (!entry) return undefined;
+    this.entries.delete(key);
+    this.entries.set(key, entry);
+    return structuredClone(entry.value);
+  }
+  set(text, value) {
+    const bytes = Buffer.byteLength(text);
+    if (bytes > this.maxBytes) return;
+    const key = sha256Hex("json@1\0" + text);
+    if (this.entries.has(key)) this.bytes -= this.entries.get(key).bytes;
+    this.entries.delete(key);
+    while ((this.bytes + bytes > this.maxBytes || this.entries.size >= 1000) && this.entries.size) {
+      const oldest = this.entries.keys().next().value;
+      this.bytes -= this.entries.get(oldest).bytes;
+      this.entries.delete(oldest);
+    }
+    this.entries.set(key, { bytes, value: structuredClone(value) });
+    this.bytes += bytes;
+  }
+}

package/core/codes.mjs

@@ -2,6 +2,17 @@
 // produce is named here so tests, CI and the skill can match on identity
 // rather than on human-readable message text.
 export const CODES = {
+  // Task context, session evidence and snapshot capture.
+  CONTEXT_REQUEST: "AGENTDOC_CONTEXT_REQUEST",
+  CONTEXT_BUDGET: "AGENTDOC_CONTEXT_BUDGET",
+  CONTEXT_TOKENIZER: "AGENTDOC_CONTEXT_TOKENIZER",
+  CONTEXT_COMPILE: "AGENTDOC_CONTEXT_COMPILE",
+  CONTEXT_REFERENCE: "AGENTDOC_CONTEXT_REFERENCE",
+  CONTEXT_STALE: "AGENTDOC_CONTEXT_STALE",
+  SESSION_RESULT: "AGENTDOC_SESSION_RESULT",
+  SYMBOL_REQUEST: "AGENTDOC_SYMBOL_REQUEST",
+  SNAPSHOT_UNSTABLE: "AGENTDOC_SNAPSHOT_UNSTABLE",
+
   // source & schema
   UTF8: "AGENTDOC_UTF8",
   YAML_PARSE: "AGENTDOC_YAML_PARSE",

package/core/compile.mjs

@@ -49,7 +49,27 @@ function earlyOut(res, extra = {}) {
 }
 
 export function compile(root, opts = {}) {
+  const started = performance.now();
+  const result = compileOnce(root, opts);
+  if (opts.telemetry) result.telemetry = {
+    durationMs: performance.now() - started,
+    ...result.repo?.metrics,
+    serializedBytes: result.serialized ? Buffer.byteLength(result.serialized) : 0,
+    heapUsedBytes: process.memoryUsage().heapUsed,
+    stages: result.phaseTimings || {},
+  };
+  return result;
+}
+
+function compileOnce(root, opts = {}) {
   const res = new CompileResult();
+  let stageStart = opts.telemetry ? performance.now() : 0;
+  const stage = (name) => {
+    if (!opts.telemetry) return;
+    const now = performance.now();
+    (res.phaseTimings ||= {})[name] = now - stageStart;
+    stageStart = now;
+  };
   // A peek load, only to read the generated-directory list: a project that keeps
   // build output in a directory this build does not know about must be able to
   // say so.
@@ -60,7 +80,7 @@ export function compile(root, opts = {}) {
   } catch {
     // No configuration, or an invalid one: the real load below reports it.
   }
-  const repo = new Repo(root, { noGit: opts.noGit, generatedDirs: cfgGeneratedDirs });
+  const repo = new Repo(root, { noGit: opts.noGit, generatedDirs: cfgGeneratedDirs, snapshotReads: opts.snapshotReads, parseCache: opts.parseCache });
   let bundle;
   let schemaInputs;
   try {
@@ -83,6 +103,7 @@ export function compile(root, opts = {}) {
   // a coverage failure is what scaffolding exists to fix.
   res.repo = repo;
   res.cfg = cfg;
+  stage("configuration");
   const refs = makeRefs(cfg.namespace);
   const graphPath = cfg.output.graph;
 
@@ -145,6 +166,7 @@ export function compile(root, opts = {}) {
     throw e;
   }
   const discovery = discoverUnits(repo, cfg, adapters);
+  stage("sourcesAndDiscovery");
   repo.nestedUnitRoots = discovery.nestedRoots;
   // Eligibility is the compiler's own answer to "which directories are units" —
   // exposed on the result so scaffold can write descriptors for exactly these
@@ -789,6 +811,7 @@ export function compile(root, opts = {}) {
     }
   }
 
+  stage("extractionAndAuthority");
   const graph = assembleGraph(sources, rel, indexes, derivedMap, facts, rawConflicts, observationOut, provFinal, diagnostics);
   graph.compiler = { name: COMPILER_NAME, version: COMPILER_VERSION, adapters: adapterStamp(adapters) };
 
@@ -865,6 +888,7 @@ export function compile(root, opts = {}) {
     commit,
     dirty,
   };
+  stage("assemblyAndValidation");
   return res;
 }
 

package/core/context/index.mjs

@@ -0,0 +1,173 @@
+import { CODES } from "../codes.mjs";
+// Task context protocol. Graph authority stays in the compiler; this module
+// selects and transports facts without turning an interpretation into a fact.
+import fs from 'node:fs';
+import path from 'node:path';
+import { createHash, randomUUID } from 'node:crypto';
+import { getEncoding } from 'js-tiktoken';
+import { compileSnapshot } from '../snapshot.mjs';
+import { ContextIndex, queryContext } from '../query.mjs';
+import { impact } from '../impact.mjs';
+import { canonicalPath } from '../paths.mjs';
+import { observationFreshness } from '../observations.mjs';
+import { assertNoSecretsInText } from '../secrets.mjs';
+
+const VERSION = 'agentdoc.dev/context/v1';
+const INTENTS = ['implement', 'change-contract', 'debug-runtime', 'review'];
+const ENCODERS = new Map();
+const hash = value => createHash('sha256').update(JSON.stringify(value)).digest('hex');
+function error(code, message) { throw Object.assign(new Error(message), { code }); }
+function closed(value, keys, name) {
+  if (!value || typeof value !== 'object' || Array.isArray(value)) error(CODES.CONTEXT_REQUEST, name + ' must be an object');
+  for (const k of Object.keys(value)) if (!keys.includes(k)) error(CODES.CONTEXT_REQUEST, 'unknown ' + name + ' field: ' + k);
+}
+export function validateRequest(request) {
+  closed(request, ['task', 'paths', 'intent', 'budget', 'cache'], 'request');
+  if (typeof request.task !== 'string' || !request.task.trim()) error(CODES.CONTEXT_REQUEST, 'task must be a nonempty string');
+  if (!Array.isArray(request.paths) || !request.paths.length || request.paths.some(p => typeof p !== 'string' || !p || (!/^[a-z]+:[a-z0-9-]+\//.test(p) && canonicalPath(p) === null))) error(CODES.CONTEXT_REQUEST, 'paths must be repository paths or entity refs');
+  if (request.intent !== undefined && !INTENTS.includes(request.intent)) error(CODES.CONTEXT_REQUEST, 'unsupported intent');
+  if (request.cache !== undefined && typeof request.cache !== 'boolean') error(CODES.CONTEXT_REQUEST, 'cache must be boolean');
+  if (request.budget !== undefined) closed(request.budget, ['tokens', 'encoding'], 'budget');
+  const budget = { tokens: 4096, encoding: 'o200k_base', ...request.budget };
+  if (!Number.isSafeInteger(budget.tokens) || budget.tokens < 256 || budget.tokens > 1000000) error(CODES.CONTEXT_BUDGET, 'tokens must be an integer from 256 to 1000000');
+  if (!['o200k_base', 'cl100k_base'].includes(budget.encoding)) error(CODES.CONTEXT_TOKENIZER, 'supported pinned encodings: o200k_base, cl100k_base');
+  return { task: request.task.trim(), paths: [...new Set(request.paths)].sort(), intent: request.intent || 'implement', budget, cache: request.cache !== false };
+}
+export function countTokens(text, encoding = 'o200k_base') {
+  if (!ENCODERS.has(encoding)) ENCODERS.set(encoding, getEncoding(encoding));
+  return ENCODERS.get(encoding).encode(text, [], []).length;
+}
+export function serializePacket(packet) { return JSON.stringify(packet) + '\n'; }
+function record(kind, subject, key, value, evidence = [], critical = false) {
+  const id = hash([kind, subject, key]).slice(0, 24);
+  return JSON.parse(JSON.stringify({ id, version: hash([value, evidence]), kind, subject, key, value, evidence: [...evidence].sort(), critical }));
+}
+function collect(res, request) {
+  const g = res.graph, idx = new ContextIndex(g), refs = new Set(), records = new Map();
+  const im = impact(g, request.paths);
+  const put = r => records.set(r.id, r);
+  for (const p of request.paths) { const e = idx.resolve(p); if (e) refs.add(e.ref); }
+  for (const e of im.affectedComponents) refs.add(e.ref);
+  // Unlimited selection here: the single serialized token budget controls delivery.
+  const unlimited = Object.fromEntries(['relations','events','externals','capabilities','verification','journeys','conflicts','assertions','provenance','contracts'].map(k => [k, Number.MAX_SAFE_INTEGER]));
+  for (const ref of [...refs].sort()) {
+    const ctx = queryContext(g, ref, { budget: unlimited, index: idx });
+    put(record('entity', ref, 'identity', { description: ctx.entity.description, derived: ctx.entity.derived }, ctx.entity.sourcePaths));
+    for (const r of ctx.relations.items) put(record('relation', ref, r.type + ':' + r.direction + ':' + r.other, r, r.provenanceIds));
+    for (const a of ctx.authority.assertions.items) put(record('assertion', ref, a.key + ':' + a.id, a, [], a.key.startsWith('requirement.')));
+    for (const c of ctx.authority.conflicts.items) put(record('conflict', ref, c.id, c, [], true));
+    for (const [kind, paths] of Object.entries(ctx.constraints)) for (const p of paths) {
+      const bytes = res.repo.readBytes(p);
+      const expected = res.graph.provenance.filter(x => x.path === p).map(x => x.contentHash.replace(/^sha256:/, ''));
+      if (!expected.includes(createHash('sha256').update(bytes).digest('hex'))) error(CODES.CONTEXT_STALE, 'document is not pinned by this snapshot: ' + p);
+      const source = new TextDecoder('utf-8', { fatal: true }).decode(bytes).replace(/^\uFEFF/, '');
+      put(record(kind === 'constraints' ? 'requirement' : 'document', ref, p,
+        { path: p, sha256: createHash('sha256').update(bytes).digest('hex'), text: source }, [p], kind === 'constraints'));
+    }
+    for (const c of ctx.contracts.items) put(record('contract', ref, c.id || c.ref, c, c.provenanceIds));
+    for (const o of ctx.authority.observations) put(record('observation', ref, o.id, o, o.evidenceBundle ? [o.evidenceBundle] : [], true));
+    for (const r of ctx.reviewCommitments.filter(r => !r.subject || refs.has(r.subject))) put(record('review', r.subject || 'repository', r.key || r.kind, r, [], true));
+    for (const j of ctx.journeys.items) put(record('journey', ref, j.id, j));
+    for (const e of ctx.events.items) put(record('event', ref, e.pattern, e, [], true));
+  }
+  const checks = new Map();
+  for (const p of [...refs]) {
+    const ctx = queryContext(g, p, { budget: unlimited, index: idx });
+    for (const v of [...ctx.verification.items, ...ctx.globalGates]) checks.set(v.id, v);
+  }
+  for (const v of im.recommendedVerification) checks.set(v.id, v);
+  for (const v of checks.values()) {
+    const original = [...g.verification, ...g.gates].find(x => x.id === v.id);
+    const owners = original?.componentRefs || [];
+    // Command text alone cannot prove a working directory; keep that unknown.
+    put(record('check', 'repository', v.id, { ...v, componentRefs: owners, execution: { kind: 'shell', cwd: null, status: 'requires-context' }, status: 'recommended' }, original?.provenanceIds, true));
+  }
+  for (const f of observationFreshness(g.observations)) if (f.stale) put(record('freshness', 'repository', f.id, f, [], true));
+  const sort = (a, b) => Number(b.critical) - Number(a.critical) || a.kind.localeCompare(b.kind) || a.id.localeCompare(b.id);
+  return { records: [...records.values()].sort(sort), scope: [...refs].sort(), coverage: { unmatched: im.unmatchedFiles, limitations: ['component-level analysis; dynamic and unsupported relationships may be absent'], staleObservations: observationFreshness(g.observations).filter(f => f.stale).map(f => f.id) } };
+}
+function storeDir(root) { return path.join(root, '.agentdoc', 'context'); }
+function save(root, id, data) {
+  const dir = storeDir(root); fs.mkdirSync(dir, { recursive: true });
+  const temp = path.join(dir, id + '.' + randomUUID() + '.tmp');
+  fs.writeFileSync(temp, JSON.stringify({ identity: data.identity }), { mode: 0o600 });
+  fs.renameSync(temp, path.join(dir, id + '.json'));
+  const retained = fs.readdirSync(dir).filter(n => /^[a-f0-9]{64}\.json$/.test(n)).map(n => ({ n, time: fs.statSync(path.join(dir,n)).mtimeMs })).sort((a,b) => b.time-a.time || a.n.localeCompare(b.n));
+  for (const old of retained.slice(64)) if (old.n !== id + '.json') fs.rmSync(path.join(dir,old.n), { force: true });
+}
+function load(root, id) {
+  if (!/^[a-f0-9]{64}$/.test(id || '')) error(CODES.CONTEXT_REFERENCE, 'invalid packet reference');
+  try { const x = JSON.parse(fs.readFileSync(path.join(storeDir(root), id + '.json'), 'utf8')); if (hash(x.identity) !== id) throw new Error(); return { ...x.identity, id };  }
+  catch { error(CODES.CONTEXT_REFERENCE, 'packet unavailable; run prepare again'); }
+}
+function packetPage(state, offset, limit) {
+  const p = { schemaVersion: VERSION, packet: state.id, snapshot: state.snapshot, task: state.request.task, intent: state.request.intent, scope: state.scope, coverage: state.coverage,
+    records: [], delivery: { offset, total: state.records.length, omitted: state.records.length - offset, criticalOmitted: 0, partial: true },
+    continuation: { packet: state.id, offset }, budget: { limit, used: 0, encoding: state.request.budget.encoding, tokenizerVersion: 'js-tiktoken@1.0.21' } };
+  const measure = () => {
+    // Digit count feeds back into encoding; converge to the exact final size.
+    for (let i = 0; i < 12; i++) { const n = countTokens(serializePacket(p), p.budget.encoding); if (n === p.budget.used) return n; p.budget.used = n; }
+    error(CODES.CONTEXT_BUDGET, 'token accounting did not converge');
+  };
+  const update = () => {
+    const next = offset + p.records.length;
+    p.delivery.omitted = state.records.length - next;
+    p.delivery.criticalOmitted = state.records.slice(next).filter(r => r.critical).length;
+    p.delivery.partial = next < state.records.length;
+    p.continuation = next < state.records.length ? { packet: state.id, offset: next } : null;
+    return measure();
+  };
+  if (update() > limit) error(CODES.CONTEXT_BUDGET, 'budget cannot hold the minimum envelope');
+  for (const r of state.records.slice(offset)) {
+    p.records.push(r);
+    if (update() > limit) { p.records.pop(); update(); break; }
+  }
+  if (!p.records.length && offset < state.records.length) error(CODES.CONTEXT_BUDGET, 'next record exceeds budget; expand with a larger token limit');
+  assertNoSecretsInText(serializePacket(p), 'context packet');
+  return p;
+}
+export function prepare(root, input, options = {}) {
+  const request = validateRequest(input);
+  const res = compileSnapshot(root, { cache: request.cache, ...options });
+  if (res.errors.length) error(CODES.CONTEXT_COMPILE, res.errors.map(e => e.message).join('\n'));
+  const selected = collect(res, request);
+  const snapshot = { inputHash: res.stats.inputHash, compiler: res.graph.compiler, graphHash: hash(res.graph) };
+  const identity = { snapshot, request, records: selected.records, logicalRecords: selected.records, scope: selected.scope, coverage: selected.coverage };
+  assertNoSecretsInText(JSON.stringify(identity), 'context state');
+  const id = hash(identity);
+  const state = { identity, id, snapshot, request, ...selected };
+  const packet = packetPage(state, 0, request.budget.tokens);
+  save(root, id, state);
+  return packet;
+}
+export function expand(root, request) {
+  closed(request, ['packet', 'offset', 'tokens'], 'expand');
+  const state = load(root, request.packet);
+  const offset = request.offset ?? 0, tokens = request.tokens ?? state.request.budget.tokens;
+  if (!Number.isSafeInteger(offset) || offset < 0 || offset > state.records.length || !Number.isSafeInteger(tokens) || tokens < 256 || tokens > 1000000) error(CODES.CONTEXT_REQUEST, 'invalid offset or token limit');
+  const res = compileSnapshot(root, { cache: state.request.cache });
+  if (res.errors.length || hash(res.graph) !== state.snapshot.graphHash || JSON.stringify(observationFreshness(res.graph.observations).filter(f => f.stale).map(f => f.id)) !== JSON.stringify(state.coverage.staleObservations)) error(CODES.CONTEXT_STALE, 'snapshot changed; use refresh or prepare');
+  return packetPage(state, offset, tokens);
+}
+export function refresh(root, request) {
+  closed(request, ['packet', 'acknowledged', 'reset'], 'refresh');
+  const previous = load(root, request.packet);
+  if (request.reset !== undefined && typeof request.reset !== 'boolean') error(CODES.CONTEXT_REQUEST, 'reset must be boolean');
+  if (request.acknowledged !== undefined && (!Array.isArray(request.acknowledged) || request.acknowledged.some(x => typeof x !== 'string'))) error(CODES.CONTEXT_REQUEST, 'acknowledged must be record IDs');
+  const known = new Set(request.reset ? [] : request.acknowledged || []);
+  for (const id of known) if (!(previous.logicalRecords || previous.records).some(r => r.id === id)) error(CODES.CONTEXT_REQUEST, 'acknowledged ID is not in the previous packet');
+  const initial = prepare(root, previous.request);
+  const current = load(root, initial.packet);
+  const old = new Map((previous.logicalRecords || previous.records).filter(r => known.has(r.id)).map(r => [r.id, r]));
+  const currentIds = new Set(current.records.map(r => r.id));
+  const records = current.records.filter(r => r.critical || !old.has(r.id) || old.get(r.id).version !== r.version);
+  for (const [id, r] of old) if (!currentIds.has(id)) records.push(record('retraction', r.subject, id, { id, previousVersion: r.version }, [], true));
+  const state = { ...current, records };
+  // A delta is itself addressed and can be paged; acknowledge only IDs delivered.
+  state.identity = { base: previous.id, snapshot: current.snapshot, request: current.request, records, logicalRecords: current.logicalRecords || current.records, scope: current.scope, coverage: current.coverage };
+  state.id = hash(state.identity); save(root, state.id, state);
+  return packetPage(state, 0, current.request.budget.tokens);
+}
+export function forget(root, packet) {
+  load(root, packet); fs.unlinkSync(path.join(storeDir(root), packet + '.json'));
+}

package/core/context/symbols.mjs

@@ -0,0 +1,81 @@
+import { CODES } from "../codes.mjs";
+// On-demand lexical experiment. These are source navigation candidates, never
+// complete call/dependency analysis and never grounds for removing checks.
+import fs from "node:fs";
+import path from "node:path";
+import { createHash } from "node:crypto";
+import { canonicalPath } from "../paths.mjs";
+
+function reject(message) { throw Object.assign(new Error(message), { code: CODES.SYMBOL_REQUEST }); }
+// Preserve source offsets while hiding comments and quoted text. Interpolated
+// template expressions and regex literals are deliberately unsupported.
+function codeOnly(text) {
+  let out = "", state = "code";
+  for (let i = 0; i < text.length; i++) {
+    const c = text[i], next = text[i + 1];
+    if (state === "code") {
+      if (c === "/" && next === "/") { state = "line"; out += "  "; i++; }
+      else if (c === "/" && next === "*") { state = "block"; out += "  "; i++; }
+      else if (c === "'" || c === '"' || c === "`") { state = c; out += " "; }
+      else out += c;
+    } else if (state === "line") {
+      out += c === "\n" ? "\n" : " ";
+      if (c === "\n") state = "code";
+    } else if (state === "block") {
+      if (c === "*" && next === "/") { state = "code"; out += "  "; i++; }
+      else out += c === "\n" ? "\n" : " ";
+    } else if (c === "\\") {
+      out += " ";
+      if (next !== undefined) { out += next === "\n" ? "\n" : " "; i++; }
+    } else {
+      out += c === "\n" ? "\n" : " ";
+      if (c === state) state = "code";
+    }
+  }
+  return out;
+}
+
+export function indexTypeScriptSymbols(root, files) {
+  if (!Array.isArray(files) || !files.length || files.some((f) => typeof f !== "string")) reject("files must be a nonempty list of TypeScript paths");
+  const base = fs.realpathSync(root);
+  const paths = files.map((file) => {
+    const rel = canonicalPath(file);
+    if (rel === null || !/\.(?:[cm]?ts|tsx)$/.test(rel)) reject("only repository-relative TypeScript files are supported");
+    return rel;
+  });
+  const sources = [...new Set(paths)].sort().map((file) => {
+    let absolute, source;
+    try {
+      absolute = fs.realpathSync(path.join(base, file));
+      if (!absolute.startsWith(base + path.sep)) reject("source escapes repository");
+      source = fs.readFileSync(absolute, "utf8");
+    } catch (e) {
+      if (e.code === CODES.SYMBOL_REQUEST) throw e;
+      reject("source unavailable: " + file);
+    }
+    const clean = codeOnly(source);
+    const declarations = [];
+    const pattern = /\b(export\s+)?(?:default\s+)?(?:declare\s+)?(?:abstract\s+)?(?:async\s+)?(function|class|interface|type|enum|const|let|var)\s+([A-Za-z_$][\w$]*)/g;
+    for (const match of clean.matchAll(pattern)) {
+      const offset = match.index + match[0].lastIndexOf(match[3]);
+      const before = source.slice(0, offset);
+      declarations.push({ name: match[3], kind: match[2], exported: Boolean(match[1]),
+        line: before.split("\n").length, column: offset - before.lastIndexOf("\n") });
+    }
+    return { path: file, sha256: createHash("sha256").update(source).digest("hex"), declarations, clean };
+  });
+  const names = new Set(sources.flatMap((s) => s.declarations.map((d) => d.name)));
+  const indexed = sources.map(({ clean, ...file }) => {
+    const references = [];
+    for (const match of clean.matchAll(/[A-Za-z_$][\w$]*/g)) {
+      if (!names.has(match[0])) continue;
+      const before = clean.slice(0, match.index);
+      const line = before.split("\n").length, column = match.index - before.lastIndexOf("\n");
+      const declaration = file.declarations.some((d) => d.line === line && d.column === column);
+      if (!declaration) references.push({ name: match[0], line, column });
+    }
+    return { ...file, references };
+  });
+  return { schemaVersion: "agentdoc.dev/symbol-candidates/v1", coverage: { partial: true, checkSelection: "unchanged",
+    limitations: ["lexical candidates only; scope, imports and overloads are not resolved", "dynamic uses, template interpolation and regex literals are not resolved", "coverage is restricted to the explicitly requested files"] }, files: indexed };
+}