@hachej/boring-agent 0.4.6 → 0.4.7

package/README.md

@@ -91,7 +91,7 @@ The host supplies byte access and optional OCR; this package performs no file/ne
 
 Results use native `output` for the model and the same metadata in `details`. As in the reference, typed refusals are ordinary results carrying `error`, not `isError: true`. Document content is untrusted evidence, never instructions or consent. The readers set Pi's native output byte limit to 8 MiB to retain valid JSON at the extraction caps; the JSON occupies one native output line. These are returned-extraction limits, not bounds on input bytes or parser memory/CPU. Enforce input and resource budgets in the host.
 
-The regular dependencies are pinned `unpdf@1.9.1` (MIT) and the official SheetJS `xlsx@0.20.3` HTTPS tarball (Apache-2.0), not npm's old `xlsx@0.18.5`. Both are imported lazily inside their reader only. Their notices ship in `THIRD_PARTY_NOTICES.md`. `test/packages/agent-documents.test.mjs` drives public registrations through a real Harness with generated fictional files; no live OCR provider is qualified.
+PDF reading uses the pinned regular dependency `unpdf@1.9.1` (MIT). Spreadsheet reading needs the optional peer `xlsx` (SheetJS, Apache-2.0); install the official distribution in the host with `npm i https://cdn.sheetjs.com/xlsx-0.20.3/xlsx-0.20.3.tgz`, not npm's old `xlsx@0.18.5`. Without it, the spreadsheet tool returns a clear refusal; PDF reading still works. Both packages are imported lazily inside their reader only. Their notices ship in `THIRD_PARTY_NOTICES.md`. `test/packages/agent-documents.test.mjs` drives public registrations through a real Harness with generated fictional files, and `npm run test:question-consumer` exercises the packed agent without SheetJS; no live OCR provider is qualified.
 
 ## Approval gate
 
@@ -114,7 +114,7 @@ await conversation.configure({ extensions: { add: [memory.extension] } }, contex
 await conversation.configure({ extensions: { remove: [memory.extension] } }, context); // off again
 ```
 
-Pi applies an extension's hooks only in conversations that select it, so memory is opt-in per conversation and there is no separate mode. A host that selects extensions by default selects it everywhere; give conversations an explicit `extensions` list to keep it opt-in. `createOptChatMemory(options)` returns `{ extension, nap(id), stats(id), zoom(id, leafId, n), date(id, leafId), forgetMessage(id, messageIndex), forget(id), dispose() }`. Options: `harness`, `context` (long-lived, for background work), `summarizer` (required: `{ model }`, where `model` may be a function `(conversationId) => ModelRef`, sync or async, for example that conversation's own model, and `models` defaults to the Harness's; or `{ summarize(job, signal) }` returning one line), `nodeBytes` (512), `viewBytes` (128000), `compactBytes` (32000, the compactor's separate view), `foldHighWater` (1.25) and `foldLowWater` (0.75), fractions of both view budgets, `notes` (seed strings, stored once per conversation before the transcript), `workOf(message)` (host classification of subagent or job reports), `jobs` (64, how many summary tasks of a conversation the host keeps live at once), `tries` (5), `cap` (30000), `retryMs` (10000 initially, doubling after each failure to at most 60000), `settleTimeoutMs` (60000), `agentName`, `name`, `onError`. `dispose()` only stops this process's waits: the summary tasks belong to Pi, which keeps and recovers them, so it is harmless to skip and safe to call before `harness.close`.
+Pi applies an extension's hooks only in conversations that select it, so memory is opt-in per conversation and there is no separate mode. A host that selects extensions by default selects it everywhere; give conversations an explicit `extensions` list to keep it opt-in. `createOptChatMemory(options)` returns `{ extension, nap(id), stats(id), zoom(id, leafId, n), date(id, leafId), forgetMessage(id, messageIndex), forget(id), dispose() }`. Options: `harness`, `context` (long-lived, for background work), `summarizer` (required: `{ model }`, where `model` may be a function `(conversationId) => ModelRef`, sync or async, for example that conversation's own model, and `models` defaults to the Harness's; or `{ summarize(job, signal) }` returning one line), `nodeBytes` (512), `viewBytes` (128000), `compactBytes` (32000, the cold-start/callback compactor view), `foldHighWater` (1.25) and `foldLowWater` (0.75), fractions of both view budgets, `notes` (seed strings, stored once per conversation before the transcript), `workOf(message)` (host classification of subagent or job reports), `jobs` (64, how many summary tasks of a conversation the host keeps live at once), `tries` (5), `cap` (30000), `retryMs` (10000 initially, doubling after each failure to at most 60000), `settleTimeoutMs` (60000), `agentName`, `name`, `onError`. `dispose()` only stops this process's waits: the summary tasks belong to Pi, which keeps and recovers them, so it is harmless to skip and safe to call before `harness.close`.
 
 | Reference | Here |
 | --- | --- |
@@ -141,7 +141,8 @@ Deviations, honestly:
 - The log is Pi's model context, not an OptChat file, so ids are positions in that context. A native compaction, reset or edit that changes earlier messages invalidates the summaries it touches (fingerprints), and the view is refolded. `beforeCompact` declines while the extension is selected: the view replaces compaction, and a native summary would cut the transcript the view is built from. That is the only effect on Pi's lifecycle.
 - The current turn is verbatim (including its tool steps) instead of being summarized to one line each as soon as it is logged; it enters the view when the next user message arrives. A user message sent mid-run starts a new "current turn" for the next step.
 - A request is rewritten once per attempt from the full context. The summaries are a cache rebuilt from the log; the summarizer's calls are not Pi generations: they run as Pi tasks and their usage is in `stats().compactor.usage`, not `pi.usage` (see above).
-- Both views use four-line text blocks with a stable prefix. The compactor folds its own `compactBytes` view, independent of the chat's `viewBytes`. Earlier summary jobs take only lines wholly before their source from that same tiling, rather than unfolding an older checkpoint. Admission builds preceding parents before dispatching later requests whose context would exceed this budget. Pi 1.2's `Models.complete` accepts retention preferences, but `TextContent` exposes no per-block cache marker. Blocks are unmarked; provider-specific cache-hit guarantees are unavailable. The 600-message/100-turn prefix regression uses 128 KB for both views. Smaller compactor budgets spend a larger share on uncached source/scale text; a stable fold alone does not guarantee 95% cache reads for complete requests. Qualify the host's chosen budgets with its provider. There is no separate `fsync` log or single-writer lock; Pi's storage owns durability. Summaries are read from documents on every request and stats call.
+- Both views use four-line text blocks. Model compactor requests carry the entire last emitted turn view (bare lines), even when their job revisits older history; the admitted `upTo` still bounds the source they may summarize. Before the first turn, native documents retain one frozen, whole-block prefix within `compactBytes × foldHighWater`. Callback summarizers keep their past-only `compactBytes` context. The constant size example is in the summarizer system prompt rather than every job suffix. The first native task on a new conversation/model/prefix sends alone; other admitted tasks await its response, then send in parallel within the same `jobs` cap. Failure or cancellation releases the warmup wait. This is process-local request coordination, not a scheduler or proof a provider retained its cache.
+- `stats().compactor.requests` exposes provider-reported input, cache reads/writes, output and cost for the most recent 256 responses booked by settled native tasks, including retries, oldest first. The history limit is `COMPACTOR_REQUEST_HISTORY_LIMIT`, next to `DEFAULTS`; lifetime `calls` and aggregate `usage` totals are not truncated. These records survive restart; old state documents start with no request records. Interrupted tasks can lose uncommitted usage, as before. Pi 1.2 exposes no per-block cache marker: the fictional Anthropic replay supplies host breakpoints and measures cold starts plus 100 turns at 600/full-fold and 400/default messages. It enforces ≥95% cache reads, fewer uncached tokens and no extra summarizer calls; this is not live-provider qualification. Longer cached context increases total input. Qualify the host's chosen budgets, breakpoints and retention with its provider. There is no separate `fsync` log or single-writer lock; Pi's storage owns durability. Summaries are read from documents on every request and stats call.
 - Cost: the compactor makes about two calls per message, each carrying its small view as context. Choose a cheap summarizer model; the browser-agent example uses the chat model only because it has one.
 - The package runs in Node and in a browser (no Node-only imports). Fictional-data tests are in `test/packages/agent.test.mjs`; `examples/bot` and `examples/browser-agent` have journeys.
 
@@ -149,7 +150,7 @@ Deviations, honestly:
 
 `await memory.forgetMessage(conversationId, messageIndex)` forgets one message in Pi's current model context. `messageIndex` is its zero-based position in `(await conversation.context(context)).messages`, including system-message positions. System messages cannot be forgotten. This is distinct from the leaf id used by `zoom`; seeded notes and assistant tool calls can shift leaf ids.
 
-The operation replaces every leaf of that message with `(forgotten)` at the same positions. One native commit stores the tombstone and deletes those leaf summaries, their ancestors, later summaries and corresponding failure records. Later summaries can contain facts resolved from earlier context, so deleting only the logarithmic structural ancestors would leave copies behind. Rebuilding can therefore cost more than logarithmic work. Later leaves keep their ids and fingerprints. Native tasks rebuild the invalidated summaries. In-flight calls recheck the memory revision inside their commit and cannot restore old output. A chat request waiting on summaries also rechecks that revision before returning its rewritten messages. `nap` waits for rebuilding. Repeated forgetting is harmless. `forget(id)` only evicts the process cache and does not undo tombstones.
+The operation replaces every leaf of that message with `(forgotten)` at the same positions. One native commit stores the tombstone and deletes those leaf summaries, their ancestors, later summaries and corresponding failure records. Once a model summarizer has used the shared prefix, forgetting conservatively deletes all summaries: earlier nodes and transitive parents can also copy later facts. That fact is retained across restart and summarizer configuration changes. With only callback summaries, later summaries can contain facts resolved from earlier context, so deleting only the logarithmic structural ancestors would leave copies behind. Rebuilding can therefore cost more than logarithmic work. Later leaves keep their ids and fingerprints. Native tasks rebuild the invalidated summaries. In-flight calls recheck the memory revision inside their commit and cannot restore old output. A chat request waiting on summaries also rechecks that revision before returning its rewritten messages. `nap` waits for rebuilding. Repeated forgetting is harmless. `forget(id)` only evicts the process cache and does not undo tombstones.
 
 This changes active memory only, not requests already dispatched. Pi's session entries, native task history, underlying storage pages, provider logs, backups and copies quoted in other messages or existing forks are not erased. Indices address the current context, so a reset or history replacement requires the host to resolve positions again. Hosts must authorize the operation. Seed notes are stored once in `boring.memory.optchat.notes`; changing `options.notes` does not overwrite an existing conversation's notes.
 

package/dist/documents.js

@@ -147,7 +147,13 @@ async function readSpreadsheet(path, bytes, sheet, range) {
     const type = sniff(bytes, path);
     if (!SPREADSHEET_TYPES.includes(type))
         return refused(path, type, bytes.byteLength, 'Not a spreadsheet');
-    const XLSX = await import('xlsx');
+    let XLSX;
+    try {
+        XLSX = await import('xlsx');
+    }
+    catch {
+        return refused(path, type, bytes.byteLength, 'Spreadsheet reading needs the optional package xlsx (SheetJS); install it in the host');
+    }
     let book;
     try {
         book = type === TYPES.csv ? XLSX.read(textOf(bytes), { type: 'string', raw: true }) : XLSX.read(bytes, { type: 'array', cellDates: true });

package/dist/memory/optchat-core.d.ts

@@ -20,6 +20,7 @@ export declare const DEFAULTS: {
     /** How long a request waits for the summaries it needs. */
     readonly settleTimeoutMs: 60000;
 };
+export declare const COMPACTOR_REQUEST_HISTORY_LIMIT = 256;
 /** What a view line shows for a message that has no summary yet. Only a fail-safe: requests wait for summaries. */
 export declare const PLACEHOLDER = "(not summarized yet: zoom it)";
 /** UTF-8 length without allocating. */

package/dist/memory/optchat-core.js

@@ -19,6 +19,7 @@ export const DEFAULTS = {
     /** How long a request waits for the summaries it needs. */
     settleTimeoutMs: 60_000,
 };
+export const COMPACTOR_REQUEST_HISTORY_LIMIT = 256;
 /** What a view line shows for a message that has no summary yet. Only a fail-safe: requests wait for summaries. */
 export const PLACEHOLDER = '(not summarized yet: zoom it)';
 /** UTF-8 length without allocating. */

package/dist/memory/optchat.d.ts

@@ -39,6 +39,8 @@ type Failure = {
 type StateDoc = {
     calls: number;
     usage: Ledger;
+    requests?: Ledger[];
+    sharedContext?: boolean;
     failures: Record<string, Failure>;
 };
 /** Compactor bookkeeping of one conversation: calls, spend and the last failure of each node, including its retry streak. */
@@ -81,7 +83,7 @@ export interface OptChatOptions {
     readonly agentName?: string;
     readonly nodeBytes?: number;
     readonly viewBytes?: number;
-    /** Target budget of the compactor's separate checkpoint-folded view. Default 32 000 UTF-8 bytes. */
+    /** Cold-start/callback context budget. Model jobs otherwise reuse the last turn view. Default 32 000 UTF-8 bytes. */
     readonly compactBytes?: number;
     /** Checkpoint high/low marks as fractions of both view budgets. Defaults 1.25 / 0.75; set both to 1 for strict budgets. */
     readonly foldHighWater?: number;
@@ -135,6 +137,8 @@ export interface OptChatStats {
         readonly error?: string;
         /** Summarizer calls made for this conversation (from the state document, so it survives a restart). */
         readonly calls: number;
+        /** Latest 256 provider response usage records booked by settled tasks, including retries; retained across restart. Lifetime calls/usage are not truncated. Interrupted tasks and callback summarizers have no usage record. */
+        readonly requests: readonly Readonly<Ledger>[];
         /** Spend of those calls (kept by this extension; not in Pi's `pi.usage`). */
         readonly usage: {
             readonly input: number;

package/dist/memory/optchat.js

@@ -10,7 +10,7 @@
 import { CompactionTask, GenerationTask, defineDoc, defineDocFamily, defineExtension, defineTask, defineTool, hook, section } from '@earendil-works/pi-durable';
 import { withAbortSignal, withCancel } from '@earendil-works/chord/context';
 import { Type } from '@earendil-works/pi-ai';
-import { DEFAULTS, OptChatLog, OptChatTree, advanceFold, bareLines, byteLength, compactPrompt, cutBytes, dateText, emptyFold, leavesOf, nodeKey, openParts, partStart, renderView, scaleLine, span, viewDoc, zoomText, } from './optchat-core.js';
+import { COMPACTOR_REQUEST_HISTORY_LIMIT, DEFAULTS, OptChatLog, OptChatTree, advanceFold, bareLines, byteLength, compactPrompt, cutBytes, dateText, emptyFold, leavesOf, nodeKey, openParts, partStart, renderView, scaleLine, span, viewDoc, zoomText, } from './optchat-core.js';
 export { DEFAULTS as OPTCHAT_DEFAULTS, PLACEHOLDER as OPTCHAT_PLACEHOLDER, compactPrompt, viewDoc } from './optchat-core.js';
 export { OptChatLog, OptChatTree, advanceFold, emptyFold, leavesOf, renderView, zoomText, dateText } from './optchat-core.js';
 /** Summaries of one conversation, in shards of 64 nodes per level (`<level>.<shard>`). A fork starts with the current ones; fingerprints reject any that no longer match. */
@@ -125,6 +125,8 @@ export function createOptChatMemory(options) {
     const logOptions = (seed, edits) => ({ notes: seed, forgotten: edits.forgotten, ...(options.workOf ? { workOf: options.workOf } : {}) });
     const sessions = new Map();
     const naps = new Map();
+    const warming = new Map();
+    const warmed = new Map();
     const retryDelay = (failure) => Math.min(60_000, retryMs * 2 ** Math.min(30, (failure.attempts ?? 1) - 1));
     const aborter = new AbortController();
     const hostContext = withAbortSignal(aborter.signal, options.context);
@@ -162,7 +164,7 @@ export function createOptChatMemory(options) {
         const saved = await options.harness().snapshot(OptChatFold, s.id, kind, options.context);
         if (saved?.fold && saved.configuration === configuration && saved.revision === revision && saved.fold.T <= to
             && saved.fingerprints.every((fp, i) => { const part = saved.fold.parts[i]; return fp === s.log.fp(part.level, part.index); }))
-            s[kind === 'turn' ? 'fold' : 'compactFold'] = copyFold(saved.fold);
+            s[kind === 'compact' ? 'compactFold' : 'fold'] = copyFold(saved.fold);
     }
     async function saveFold(s, revision, kind, fold, context) {
         await (await conversationOf(s.id)).commit(async (tx) => {
@@ -207,7 +209,7 @@ export function createOptChatMemory(options) {
     }
     // ---- The summarizer (runs inside the native task) ----------------------------------------------------------------------
     /** One node by the reference protocol (§4.2, §4.3): context first, then the step; over-long lines are resubmitted cut at the limit; the shortest wins. */
-    async function askModel(job, runtimeModels, signal) {
+    async function askModel(job, runtimeModels, signal, prefix = job.context) {
         const summarizer = options.summarizer;
         if ('summarize' in summarizer) {
             try {
@@ -225,20 +227,53 @@ export function createOptChatMemory(options) {
         if (!model)
             throw new Error(`Summarizer model ${ref.provider}/${ref.modelId} is not available`);
         const lines = job.source.map(line => line.replace(/\r?\n/g, ' '));
-        const step = `For scale, this line is exactly ${scale.bytes} bytes:\n${scale.text}\n\n` + (job.level === 0
+        const prompt = `${system}\n\nFor scale, this line is exactly ${scale.bytes} bytes:\n${scale.text}`;
+        const step = job.level === 0
             ? `Compress this message into one line, in at most ${nodeBytes} bytes:\n${job.source[0]}`
-            : `Merge these two lines into one, in at most ${nodeBytes} bytes:\n${lines.join('\n')}`);
-        const conversation = [message([...chatBlocks(job.context), step])];
+            : `Merge these two lines into one, in at most ${nodeBytes} bytes:\n${lines.join('\n')}`;
+        const conversation = [message([...chatBlocks(prefix), step])];
         const attempts = [];
+        const requests = [];
         let usage, calls = 0, spender;
+        const owner = `${String(job.conversationId)}:${ref.provider}/${ref.modelId}`;
+        const key = JSON.stringify([owner, prefix]);
+        let release;
         try {
+            // Tasks already occupy the host's admission cap. Only the first request on a new prefix warms it.
+            while (warmed.get(owner) !== prefix) {
+                const pending = warming.get(key);
+                if (!pending) {
+                    warming.set(key, new Promise(resolve => { release = resolve; }));
+                    break;
+                }
+                if (signal.aborted)
+                    throw signal.reason;
+                let abort;
+                try {
+                    await Promise.race([pending, new Promise((_resolve, reject) => {
+                            abort = () => reject(signal.reason);
+                            signal.addEventListener('abort', abort, { once: true });
+                        })]);
+                }
+                finally {
+                    if (abort)
+                        signal.removeEventListener('abort', abort);
+                }
+            }
             for (;;) {
-                const reply = await models.complete(model, { systemPrompt: system, messages: conversation }, { signal });
+                const reply = await models.complete(model, { systemPrompt: prompt, messages: conversation }, { signal });
                 calls++;
                 spender = `${reply.provider}/${reply.model}`;
                 usage = addUsage(usage, reply.usage);
+                requests.push({ input: reply.usage.input, output: reply.usage.output, cacheRead: reply.usage.cacheRead, cacheWrite: reply.usage.cacheWrite, cost: reply.usage.cost.total });
                 if (reply.stopReason === 'error' || reply.stopReason === 'aborted')
                     throw new Error(reply.errorMessage ?? `Summarizer stopped: ${reply.stopReason}`);
+                warmed.set(owner, prefix);
+                if (release) {
+                    warming.delete(key);
+                    release();
+                    release = undefined;
+                }
                 const line = reply.content.filter(part => part.type === 'text').map(part => part.text).join('\n').trim();
                 if (!line)
                     throw new Error('The summarizer returned nothing');
@@ -251,14 +286,24 @@ export function createOptChatMemory(options) {
         catch (error) {
             if (signal.aborted)
                 throw error;
-            return { error, ...(usage ? { usage } : {}), calls, ...(spender ? { spender } : {}) };
+            return { error, ...(usage ? { usage } : {}), calls, requests, ...(spender ? { spender } : {}) };
         }
-        return { text: attempts.reduce((best, line) => byteLength(line) < byteLength(best) ? line : best), ...(usage ? { usage } : {}), calls, ...(spender ? { spender } : {}) };
+        finally {
+            if (release) {
+                warming.delete(key);
+                release();
+            }
+        }
+        return { text: attempts.reduce((best, line) => byteLength(line) < byteLength(best) ? line : best), ...(usage ? { usage } : {}), calls, requests, ...(spender ? { spender } : {}) };
     }
     /** What one node task adds to the conversation's documents in its single commit: calls, spend and the node or its failure. */
     async function bookkeeping(tx, id, summary) {
         const state = await tx.doc(OptChatState, id);
         state.calls += summary.calls;
+        if (summary.requests?.length) {
+            state.requests = [...(state.requests ?? []), ...summary.requests].slice(-COMPACTOR_REQUEST_HISTORY_LIMIT);
+            state.sharedContext = true;
+        }
         if (summary.usage && summary.spender) {
             const u = summary.usage;
             state.usage.input += u.input;
@@ -292,7 +337,7 @@ export function createOptChatMemory(options) {
                 };
                 const revision = await load();
                 const ready = () => s.log.fp(level, index) === of && (level === 0 || (s.tree.text(level - 1, 2 * index) !== undefined && s.tree.text(level - 1, 2 * index + 1) !== undefined));
-                if (!ready() || s.tree.text(level, index) !== undefined)
+                if ((index + 1) * span(level) > upTo || !ready() || s.tree.text(level, index) !== undefined)
                     return stale(); // the history changed, a child is missing, or the node needs no summary
                 const to = Math.min(upTo, index * span(level));
                 const fold = compactView(s, to);
@@ -301,9 +346,15 @@ export function createOptChatMemory(options) {
                 await saveFold(s, revision, 'compact', fold, context);
                 const source = level === 0 ? [s.log.leaves[index].line] : [s.tree.text(level - 1, 2 * index), s.tree.text(level - 1, 2 * index + 1)];
                 const job = { conversationId: id, level, index, range: [index * span(level), (index + 1) * span(level) - 1], source, context: bareLines(fold, s.tree, to), nodeBytes };
+                // Keep the callback's past-only context; model requests carry the last turn's full prefix, with the job after it.
+                await restoreFold(s, revision, s.log.leaves.length, 'bootstrap');
+                await restoreFold(s, revision, s.log.leaves.length);
+                const prefixFold = s.fold.stableThrough === undefined ? fold : s.fold;
+                await loadStored(s, runtime, context); // the selected turn may reference summaries committed since load()
+                const prefix = bareLines(prefixFold, s.tree, prefixFold.T);
                 let summary;
                 try {
-                    summary = await askModel(job, runtime.models, runtime.signal);
+                    summary = await askModel(job, runtime.models, runtime.signal, prefix);
                 }
                 catch (error) {
                     if (runtime.signal.aborted)
@@ -374,6 +425,26 @@ export function createOptChatMemory(options) {
         const failures = (await options.harness().snapshot(OptChatState, s.id, options.context))?.failures ?? {};
         const now = Date.now(), upTo = Math.min(s.upTo, s.log.leaves.length);
         const conversation = await conversationOf(s.id);
+        // Before the first turn there is no sent view. Freeze one cold prefix for the seed's native tasks.
+        if ('model' in options.summarizer) {
+            const revision = (await memoryEdits(s.id)).revision;
+            const saved = await options.harness().snapshot(OptChatFold, s.id, 'bootstrap', options.context);
+            if (!saved?.fold || saved.revision !== revision || saved.configuration !== configuration
+                || !saved.fingerprints.every((fp, i) => fp === s.log.fp(saved.fold.parts[i].level, saved.fold.parts[i].index))) {
+                const prefix = emptyFold();
+                for (let i = 0; i + 4 <= s.fold.parts.length; i += 4) {
+                    const block = s.fold.parts.slice(i, i + 4), bytes = block.reduce((sum, part) => sum + part.bytes, 0);
+                    if (prefix.size + bytes > compactBytes * highWater)
+                        break;
+                    prefix.parts.push(...block.map(part => ({ ...part })));
+                    prefix.size += bytes;
+                    const last = block.at(-1);
+                    prefix.T = partStart(last) + span(last.level);
+                }
+                prefix.stableThrough = prefix.T;
+                await saveFold(s, revision, 'bootstrap', prefix, options.context);
+            }
+        }
         let waiting = 0, created = 0;
         const live = await conversation.commit(async (tx) => {
             const found = new Map();
@@ -481,7 +552,7 @@ export function createOptChatMemory(options) {
                 skip.add(done.key);
         }
     }
-    /** Context can copy an earlier fact into a later summary. Invalidate those too, atomically with the tombstone. */
+    /** Context can copy facts across nodes. Shared-prefix model summaries may depend on any leaf, transitively. */
     async function forgetMessage(id, messageIndex) {
         if (!Number.isInteger(messageIndex) || messageIndex < 0)
             throw new Error('Invalid message index');
@@ -505,8 +576,10 @@ export function createOptChatMemory(options) {
                 doc.revision++;
                 const state = await tx.doc(OptChatState, id);
                 for (let level = 0; span(level) <= leaves.length; level++) {
-                    for (let index = Math.floor(start / span(level)); (index + 1) * span(level) <= leaves.length; index++) {
+                    for (let index = 0; (index + 1) * span(level) <= leaves.length; index++) {
                         const shard = await tx.doc(OptChatNodes, id, shardKey(level, index), 0);
+                        if (!state.sharedContext && index < Math.floor(start / span(level)))
+                            continue;
                         delete shard.nodes[String(index)];
                         delete state.failures[nodeKey(level, index)];
                     }
@@ -627,14 +700,16 @@ export function createOptChatMemory(options) {
                 leaves: s.log.leaves.length, summaries,
                 view: { bytes: fold.size, budget: viewBytes, open: openParts(fold).length,
                     parts: fold.parts.map(part => ({ id: partStart(part), n: span(part.level), level: part.level, text: part.open ? '' : s.tree.text(part.level, part.index) ?? '', open: part.open })) },
-                compactor: { busy, retrying, failed: failures.length, idle: busy === 0 && retrying === 0, ...(latest ? { error: latest.error } : {}), calls: state?.calls ?? 0, usage: { ...(state?.usage ?? { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, cost: 0 }) } },
+                compactor: { busy, retrying, failed: failures.length, idle: busy === 0 && retrying === 0, ...(latest ? { error: latest.error } : {}), calls: state?.calls ?? 0, requests: (state?.requests ?? []).slice(-COMPACTOR_REQUEST_HISTORY_LIMIT).map(request => ({ ...request })), usage: { ...(state?.usage ?? { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, cost: 0 }) } },
                 ...(s.lastRequest ? { lastRequest: s.lastRequest } : {}),
             };
         },
         async zoom(id, nth, n) { return zoomText((await refresh(id)).tree, nth, n); },
         async date(id, nth) { return dateText((await refresh(id)).log, nth); },
         forgetMessage,
-        forget(id) { sessions.delete(String(id)); },
+        forget(id) { sessions.delete(String(id)); for (const key of warmed.keys())
+            if (key.startsWith(`${String(id)}:`))
+                warmed.delete(key); },
         async dispose() { aborter.abort(); await Promise.allSettled(naps.values()); },
     };
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hachej/boring-agent",
-  "version": "0.4.6",
+  "version": "0.4.7",
   "license": "MIT",
   "private": false,
   "type": "module",
@@ -147,15 +147,18 @@
     }
   },
   "dependencies": {
-    "unpdf": "1.9.1",
+    "unpdf": "1.9.1"
+  },
+  "devDependencies": {
     "xlsx": "https://cdn.sheetjs.com/xlsx-0.20.3/xlsx-0.20.3.tgz"
   },
   "peerDependencies": {
     "@earendil-works/pi-durable": "~1.2.0",
     "@earendil-works/pi-ai": "~1.2.0",
     "@earendil-works/chord": "~1.2.0",
-    "@hachej/boring-files": "^0.4.6",
-    "@hachej/boring-ui-kit": "^0.4.6"
+    "@hachej/boring-files": "^0.4.7",
+    "@hachej/boring-ui-kit": "^0.4.7",
+    "xlsx": "^0.20.3"
   },
   "peerDependenciesMeta": {
     "@hachej/boring-files": {
@@ -163,6 +166,9 @@
     },
     "@hachej/boring-ui-kit": {
       "optional": true
+    },
+    "xlsx": {
+      "optional": true
     }
   },
   "engines": {