@hachej/boring-agent 0.4.3 → 0.4.5

package/NOTICE

@@ -0,0 +1,8 @@
+OptChat memory includes prompt wording and algorithm adaptations from
+VictorTaelin/uniichat, https://github.com/VictorTaelin/uniichat
+Reference revision: 0500bf6a8ea4bf0ce047488562022b1b6532cc30
+Copyright 2026 Higher Order Company
+Licensed under the Apache License, Version 2.0. See licenses/Apache-2.0.txt.
+
+Changed in Boring UI: Pi-native tasks and documents, Pi message kinds,
+bounded views, host-configured retries and concurrency, and memory-only forgetting.

package/README.md

@@ -8,7 +8,7 @@
 
 ## One file tool set: the guard
 
-The agent edits files with Pi's own `read`, `write`, `edit` and `bash` over the workspace's `ExecutionEnv`; there are no document or artifact tools of Boring's. `createFileGuard({ workspace, resolveAccess?, name? })` from `@hachej/boring-agent/file-guard` returns a native extension (named `name`, default `boring.files.guard`: two agents with different workspaces in one registry need two names, or the last installed guard wraps both) that wraps those three tools with the public `wrapTool` (select it after the extension that registers them). Every wrapped call runs inside the workspace provider's mutation queue (`@hachej/boring-files/workspace`), so a viewer save cannot land between `edit`'s internal read and its write. `workspace` is the call's workspace resolved like Pi's environment, or one binding `{ files, root, access? }` (see [Workspace per call](#workspace-per-call)); `resolveAccess` defaults to the binding's `access`. A genuine `read` call records the file's revision as the conversation's last-read revision of that workspace and path (`baselineKey`) in a native conversation document (`lastReadRevisions`, kept across restarts; a fork starts empty); `edit`'s internal read records nothing. `write` and `edit` of a file that exists are refused unless the conversation has read it and its revision still equals the recorded one, and the refusal tells the model to read the file again. Creating a file is allowed only while it is absent. The guard fails closed: an unresolvable workspace, a refused provider read, a path outside the root, a link that leads outside it and a path that does not exist (so Pi's read never falls back to another spelling of it, such as an NFD name) (resolved through the env's `canonicalPath`, so also a new file under a link directory) are refused for `read` as well as `write` and `edit`; nothing falls through to Pi's native tool. After a successful `write` or `edit` the baseline is the revision just written, and the guard calls the provider's `record` (owner ruling 2: observed, never receipted): the file's history gains the written revision with source `agent` (and the bytes it replaced, read before the tool ran when that revision is not already retained), and the workspace's change feed an `agent` event with the conversation. Nothing enters the publication journal. Pi's `edit` tries an exact match first and falls back to a fuzzy one (Unicode normalisation, trailing whitespace, smart quotes, dashes and special spaces); the guard does not change that. A `bash` write is not intercepted, but it changes the revision, so the next `write` or `edit` of that file is refused until it is read; it reaches the change feed (no history) through the file system's watcher or the host's `poll()` at turn end. The guard adds no strictness option and no file API of its own.
+The agent edits files with Pi's own `read`, `write`, `edit` and `bash` over the workspace's `ExecutionEnv`; Boring adds no replacement file-editing tools. The optional [document readers](#shared-document-readers) extract PDF and spreadsheet content without replacing native `read`. `createFileGuard({ workspace, resolveAccess?, name? })` from `@hachej/boring-agent/file-guard` returns a native extension (named `name`, default `boring.files.guard`: two agents with different workspaces in one registry need two names, or the last installed guard wraps both) that wraps those three tools with the public `wrapTool` (select it after the extension that registers them). Every wrapped call runs inside the workspace provider's mutation queue (`@hachej/boring-files/workspace`), so a viewer save cannot land between `edit`'s internal read and its write. `workspace` is the call's workspace resolved like Pi's environment, or one binding `{ files, root, access? }` (see [Workspace per call](#workspace-per-call)); `resolveAccess` defaults to the binding's `access`. A genuine `read` call records the file's revision as the conversation's last-read revision of that workspace and path (`baselineKey`) in a native conversation document (`lastReadRevisions`, kept across restarts; a fork starts empty); `edit`'s internal read records nothing. `write` and `edit` of a file that exists are refused unless the conversation has read it and its revision still equals the recorded one, and the refusal tells the model to read the file again. Creating a file is allowed only while it is absent. The guard fails closed: an unresolvable workspace, a refused provider read, a path outside the root, a link that leads outside it and a path that does not exist (so Pi's read never falls back to another spelling of it, such as an NFD name) (resolved through the env's `canonicalPath`, so also a new file under a link directory) are refused for `read` as well as `write` and `edit`; nothing falls through to Pi's native tool. After a successful `write` or `edit` the baseline is the revision just written, and the guard calls the provider's `record` (owner ruling 2: observed, never receipted): the file's history gains the written revision with source `agent` (and the bytes it replaced, read before the tool ran when that revision is not already retained), and the workspace's change feed an `agent` event with the conversation. Nothing enters the publication journal. Pi's `edit` tries an exact match first and falls back to a fuzzy one (Unicode normalisation, trailing whitespace, smart quotes, dashes and special spaces); the guard does not change that. A `bash` write is not intercepted, but it changes the revision, so the next `write` or `edit` of that file is refused until it is read; it reaches the change feed (no history) through the file system's watcher or the host's `poll()` at turn end. The guard adds no strictness option and no file API of its own.
 
 ## Workspace per call
 
@@ -68,6 +68,31 @@ The descriptor's `id` and `ordinal` are optional and `present` sets neither; `pa
 
 `createConvertingRead({ workspace?, resolveAccess?, convert?, cache?, maxBytes?, name? })` from `@hachej/boring-agent/file-convert` returns an extension that wraps Pi's native `read` (`wrapTool`); there is one `read`, and no second tool. Text goes to Pi's read untouched. A PDF, an office document or an image goes to the host's `convert({ name, mediaType, bytes })`, once per saved revision (cached by path and revision), and the converted text is paged the way Pi pages a text file: `offset` is the 1-based line to start at, `limit` a number of lines, a page ends at 2000 lines or 50 KB, and the note names the `offset` to continue from. What is text and what is not comes from one table, `fileKind` in `@hachej/boring-agent/file-types`, which looks at the extension and the leading bytes (Pi's read returns garbled text for a PDF, so a PDF is recognised by `%PDF-` even under another name); the mention resolver uses the same table. An archive or other binary file answers `unsupported` and is never converted; without `convert`, an image uses Pi's native behaviour and a PDF or office file answers `unsupported`. A line of converted text above 50 KB is split into 50 KB segments, each counted as one line, with a warning, so nothing is cut. The type is sniffed on the bytes the provider returns after its access check, and the converter is told the media type the file's signature names (a PDF named `.txt` arrives as `application/pdf`). The file is read through the workspace provider, so its access policy applies; a path the wrapper cannot place inside the workspace is left to the native read. Install it before `createFileGuard`, so the guard stays outermost and refuses anything outside the workspace first. `workersAiMarkdown(env.AI)` adapts a Workers AI `toMarkdown` binding to `convert` without a Cloudflare types dependency.
 
+## Shared document readers
+
+`@hachej/boring-agent/documents` exports `documentTools({ readBytes, ocr?, images? })`, `sniff(bytes, name?)` and `readStored(path, bytes, { images })`. These are native `defineTool` registrations and a native read-result helper, not a filesystem or another `read` tool. Pi's own `read` is unchanged; `file-convert` above remains the host-conversion alternative.
+
+```ts
+import { defineExtension } from '@earendil-works/pi-durable';
+import { documentTools, readStored } from '@hachej/boring-agent/documents';
+
+// readBytes is the host's authorized read through its one workspace provider.
+const tools = documentTools({ readBytes, ocr });
+registry.install(defineExtension({ name: 'app.documents', tools }));
+// A host's stored-file read path can return this native ToolExecutionResult:
+const result = readStored(path, await readBytes(path), { images: true });
+```
+
+The host supplies byte access and optional OCR; this package performs no file/network I/O and holds no credentials. Bind `readBytes(path)` to the authorized workspace/view for this conversation. Read failures propagate as native tool errors. Both readers use `replay: 'safe'`; recovery may reread bytes or call OCR again, so the host owns stable revisions, current authorization and OCR cost policy.
+
+- `read_pdf(path, pages?)` returns `{ path, totalPages, pages: [{ n, text, ocr?, truncated? }], untrusted: true }`. Page lists such as `"1-3,5"` are sorted and deduplicated, at most 30 pages per call; without a list, the first 30 pages and a `next` hint when more remain. Each page is capped at 20,000 characters. Requested pages without text go to `ocr(bytes, pageNumbers)`, which returns `{ pages?: [{ n, text? }], provider? }`. Only matching scanned pages are replaced and marked `ocr: true`; absent or failed OCR adds a note. Unlike the Factory reference's media-type argument, this callback receives the selected scanned page numbers.
+- `read_spreadsheet(path, sheet?, range?)` reads xlsx, xls, ods and CSV. It returns `sheetNames` and `{ name, rows, columns, shown, csv, truncated? }` for each selected sheet, plus `untrusted: true`. The limits are 20 sheets, 200 rows per sheet by default, 1,000 with a cell range such as `A1:C1500`, and 60,000 CSV characters per sheet. Formulas use their last computed values, never execute.
+- `sniff` uses magic numbers and ZIP entry names, not extensions (except CSV versus other UTF-8 text). `readStored` returns JSON text containing `{ path, content }` for text/CSV; images up to 4 MiB add native `ImageContent` when `images` is true. Other binaries, oversized images and image-incapable models return `{ path, type, size, error }`; PDF/spreadsheet refusals point to their reader. `images` affects this helper, not the two extraction tools.
+
+Results use native `output` for the model and the same metadata in `details`. As in the reference, typed refusals are ordinary results carrying `error`, not `isError: true`. Document content is untrusted evidence, never instructions or consent. The readers set Pi's native output byte limit to 8 MiB to retain valid JSON at the extraction caps; the JSON occupies one native output line. These are returned-extraction limits, not bounds on input bytes or parser memory/CPU. Enforce input and resource budgets in the host.
+
+The regular dependencies are pinned `unpdf@1.9.1` (MIT) and the official SheetJS `xlsx@0.20.3` HTTPS tarball (Apache-2.0), not npm's old `xlsx@0.18.5`. Both are imported lazily inside their reader only. Their notices ship in `THIRD_PARTY_NOTICES.md`. `test/packages/agent-documents.test.mjs` drives public registrations through a real Harness with generated fictional files; no live OCR provider is qualified.
+
 ## Approval gate
 
 `requireApproval(tool, { summarize })` from `@hachej/boring-agent/approval` wraps a native tool so each call waits for the person's Approve or Deny before `execute`. It reuses the ask-user mechanism (`askPerson` from `@hachej/boring-agent/ask-user`): a conversation document keyed by the tool call ID, answered with `answerUserQuestion` through the chat transport's `answer` operation, and shown by the `pi-chat` approval card. The check runs inside the tool's own execution in host code, so the model cannot skip it. While the call waits, the one-line `summarize` text is also published as the call's running details under `APPROVAL_DETAILS` (it stays as the result's details unless the tool returns its own). That is also how `pi-chat` recognizes a gated call: no list of tool names is needed. The card shows the summary as the headline with the raw arguments collapsed. Deny (or a stopped conversation) returns an error result starting `DENIED_PREFIX` and `execute` never runs. The wrapper is replay-safe: a pending approval survives a restart, and a retained native memo makes execution at most once (after a crash during execution the model gets an "outcome unknown" error instead of a repeat of the change). Reads are simply not wrapped.
@@ -80,7 +105,7 @@ Code mode: gated tools are native tools and must not be exposed to a code-mode s
 
 ## OptChat memory
 
-`@hachej/boring-agent/memory/optchat` is Victor Taelin's [OptChat](https://gist.github.com/VictorTaelin/91837951a5ce5b38f341ec1ba1df6449) (see also his [OptMem](https://github.com/VictorTaelin/OptMem)) as one native pi-durable extension: an endless conversation whose memory is the conversation itself, shown to the model as a compressed tree. Credit for the idea, the algorithm and the summarizer prompt is his; this package maps it onto Pi.
+`@hachej/boring-agent/memory/optchat` is Victor Taelin's [OptChat](https://gist.github.com/VictorTaelin/91837951a5ce5b38f341ec1ba1df6449), aligned with [uniichat at 0500bf6](https://github.com/VictorTaelin/uniichat/tree/0500bf6a8ea4bf0ce047488562022b1b6532cc30) as one native pi-durable extension: an endless conversation whose memory is the conversation itself, shown to the model as a compressed tree. Credit for the idea, the algorithm and the summarizer prompt is his; this package maps it onto Pi.
 
 ```ts
 const memory = createOptChatMemory({ harness: () => harness, context, summarizer: { models, model: { provider, modelId } } });
@@ -89,15 +114,15 @@ await conversation.configure({ extensions: { add: [memory.extension] } }, contex
 await conversation.configure({ extensions: { remove: [memory.extension] } }, context); // off again
 ```
 
-Pi applies an extension's hooks only in conversations that select it, so memory is opt-in per conversation and there is no separate mode. A host that selects extensions by default selects it everywhere; give conversations an explicit `extensions` list to keep it opt-in. `createOptChatMemory(options)` returns `{ extension, nap(id), stats(id), zoom(id, n), date(id), forget(id), dispose() }`. Options: `harness`, `context` (long-lived, for background work), `summarizer` (required: `{ model }`, where `model` may be a function `(conversationId) => ModelRef`, sync or async, for example that conversation's own model, and `models` defaults to the Harness's; or `{ summarize(job, signal) }` returning one line), `nodeBytes` (512), `viewBytes` (128000), `jobs` (8, how many summary tasks of a conversation the host keeps live at once), `tries` (5), `cap` (30000), `retryMs` (10000), `settleTimeoutMs` (60000), `agentName`, `name`, `onError`. `dispose()` only stops this process's waits: the summary tasks belong to Pi, which keeps and recovers them, so it is harmless to skip and safe to call before `harness.close`.
+Pi applies an extension's hooks only in conversations that select it, so memory is opt-in per conversation and there is no separate mode. A host that selects extensions by default selects it everywhere; give conversations an explicit `extensions` list to keep it opt-in. `createOptChatMemory(options)` returns `{ extension, nap(id), stats(id), zoom(id, leafId, n), date(id, leafId), forgetMessage(id, messageIndex), forget(id), dispose() }`. Options: `harness`, `context` (long-lived, for background work), `summarizer` (required: `{ model }`, where `model` may be a function `(conversationId) => ModelRef`, sync or async, for example that conversation's own model, and `models` defaults to the Harness's; or `{ summarize(job, signal) }` returning one line), `nodeBytes` (512), `viewBytes` (128000), `compactBytes` (32000, the compactor's separate view), `notes` (seed strings, stored once per conversation before the transcript), `workOf(message)` (host classification of subagent or job reports), `jobs` (64, how many summary tasks of a conversation the host keeps live at once), `tries` (5), `cap` (30000), `retryMs` (10000 initially, doubling after each failure to at most 60000), `settleTimeoutMs` (60000), `agentName`, `name`, `onError`. `dispose()` only stops this process's waits: the summary tasks belong to Pi, which keeps and recovers them, so it is harmless to skip and safe to call before `harness.close`.
 
 | Reference | Here |
 | --- | --- |
-| Append-only log of user, talk, tool and echo messages | The Pi transcript, never rewritten. Leaves are derived from the model context: a user message is `user`, assistant text `talk`, each tool call `tool`, each tool result `echo`; thinking is not logged. A leaf over `cap` characters keeps head and tail |
+| Append-only log of user, talk, tool and echo messages | The Pi transcript, never rewritten. Leaves are derived from the model context: a user message is `user`, assistant text `talk`, each tool call `tool`, each tool result `echo`; thinking is not logged. A leaf over `cap` characters keeps head and tail. Seeded `note` leaves precede them; messages marked by `workOf` become one `work` leaf |
 | Summary tree: node (l, i) covers messages [i·2^l, (i+1)·2^l), at most `nodeBytes` | The same, purely binary. A short message, or two children that fit together, are their own text and cost no call |
 | The view tiles the whole chat, one `id+n\|text` line per node, fit incrementally by merging the most due pair (age / 2^(level+2)), never splitting | The same fold (`advanceFold`), kept per conversation and advanced as the log grows, so the start of the view stays the same between requests. `renderView` produces the `<chat>` block |
-| Each turn calls the model with [system][view][new message] | `GenerationTask.beforeRequest` replaces the messages before the current turn with the view, as the first block of the current user message. The turn in progress (the latest user message and the steps after it) stays verbatim; every positional system message is kept in order. The explanation of the view is a prompt section, so the system prompt and tools never change between calls |
-| Compactor: context first, no ids, scale line, resubmit with the cut at the limit, `JOBS` in parallel, retry after 10 s | The same protocol (`askModel`), run by native pi-durable tasks: each node `(level, index, fingerprint)` is one `boring.memory.optchat.node` task (shipped in the extension's `tasks`, input `{ key, level, index, of, upTo }`, one `summarize` phase) that calls the summarizer through `runtime.models` (or your `summarize`) and, in one commit, writes the node and the summarizer's usage into the extension's state. Pi schedules, runs and recovers them. See below |
+| Each turn calls the model with [system][view][new message] | `GenerationTask.beforeRequest` replaces the messages before the current turn with the view, as four-line text blocks at the start of the current turn's first message. The turn in progress starts at the latest user or work message. Work reports and sibling results whose tool calls fell outside the turn become ordinary user-role input, with their full text preserved. Every positional system message is kept in order. The explanation of the view is a prompt section, so the system prompt and tools never change between calls |
+| Compactor: context first, no ids, scale line, resubmit with the cut at the limit, `JOBS` in parallel, patient retry | The same protocol (`askModel`), run by native pi-durable tasks: each node `(level, index, fingerprint)` is one `boring.memory.optchat.node` task (shipped in the extension's `tasks`, input `{ key, level, index, of, upTo }`, one `summarize` phase) that calls the summarizer through `runtime.models` (or your `summarize`) and, in one commit, writes the node and the summarizer's usage into the extension's state. Pi schedules, runs and recovers them. See below |
 | A turn waits until every line of the view is a summary and never shows cut text | The request waits up to `settleTimeoutMs`; after that the missing lines show `(not summarized yet: zoom it)`, never a clipped message, and `zoom(id, 1)` returns the message whole |
 | `zoom(id, n)` and `date(id)` | Native replay-safe tools of the same names |
 | Summary files | A conversation document family (`boring.memory.optchat.nodes`, shards of 64 nodes per level, `fork: 'current'`), one fingerprint per node over the content it covers. A fork or an edit never shows a summary of different content: a stale node is unbuilt until rebuilt |
@@ -106,8 +131,8 @@ What is native, what is host-side:
 
 - **Tasks.** The host creates node tasks with `conversation.commit(tx => tx.createTask(...))` (hooks are read-only) as `background: true`, conversation-owned tasks: they never keep the conversation busy, never block or get aborted with the user's run (`conversation.abort()` leaves them; `abort(context, { background: true })` aborts them), and show in `harness.taskGraph()` and `harness.inspect()`. A pending task after a restart is resumed by Pi once the extension is installed again (the extension carries the task definition; the options must be the same).
 - **Triggers (host admission, not scheduling).** After an answer (`onYield`, once the run is idle) and when a request needs its view (`beforeRequest`). A trigger creates the tasks of the nodes whose sources exist and whose preceding view is summarized. One live task per node key, found by scanning native tasks in the same commit that creates them. `jobs` caps how many are live per conversation; Pi has no task concurrency setting, so this is the host's admission cap and Pi's scheduler decides when they run. A node's parent is created by the next trigger or by the waiting `nap` or request, once its children are built.
-- **Waiting.** A request waits with `waitForTask` on the live node tasks, up to `settleTimeoutMs`, then shows the placeholder. `nap` waits the same way. These waits are in-process; if the process dies the tasks are not lost, only the cascade to parents waits for the next trigger.
-- **Retry.** A failed node is a failed native task (`outcome.failed`). Its last failure time is kept in the conversation document `boring.memory.optchat.state` (with the summarizer call count and spend), and a trigger creates the task again once `retryMs` has passed. No timer retries: a node in its retry window is left alone, so a request that needs it shows the placeholder instead of waiting. `onError` reports the first failure of a streak.
+- **Waiting.** A request waits with `waitForTask` on the live node tasks, up to `settleTimeoutMs`, then shows the placeholder. `nap` waits until all nodes are built, including retries. These waits are in-process; if the process dies the tasks are not lost, only the cascade to parents waits for the next trigger.
+- **Retry.** A failed node is a failed native task (`outcome.failed`). Its failure time and streak length live in `boring.memory.optchat.state` with call count and spend. The host's idle admission loop waits from `retryMs`, doubling to at most 60 seconds, then creates another native task without a new turn. `dispose()` aborts those waits, not Pi's tasks. After a process restart, `nap` or another trigger restarts admission. Requests retain their bounded wait and can show placeholders during an outage. `onError` reports only the first failure of a streak.
 - **Usage.** In the commit that stores a node (or records its failure) the task adds the summarizer's usage, all attempts of the cut-and-resubmit loop included, to the extension's own state document (`boring.memory.optchat.state`), reported by `stats().compactor.usage`. It is not written into Pi's `pi.usage`: Pi has no public way for a task to record model usage (`recordUsage` is internal), so per BORING-PI-4 that guarantee is unavailable here rather than worked around; a public usage-recording seam is an upstream request. Hosts that budget model spend add `stats().compactor.usage` to `harness.usage()`. Task and document kinds are all in `boring.memory.optchat.*`; this package writes no `pi.*` document.
 - **Still host-side.** The trigger and admission logic, the in-memory fold and log per process (a cache; fingerprints keep it honest), and the summarizer prompt. A custom `summarize` function has no model usage to record. A model call interrupted by a crash is rerun from the start of the task, and its spend is not recorded.
 
@@ -116,10 +141,20 @@ Deviations, honestly:
 - The log is Pi's model context, not an OptChat file, so ids are positions in that context. A native compaction, reset or edit that changes earlier messages invalidates the summaries it touches (fingerprints), and the view is refolded. `beforeCompact` declines while the extension is selected: the view replaces compaction, and a native summary would cut the transcript the view is built from. That is the only effect on Pi's lifecycle.
 - The current turn is verbatim (including its tool steps) instead of being summarized to one line each as soon as it is logged; it enters the view when the next user message arrives. A user message sent mid-run starts a new "current turn" for the next step.
 - A request is rewritten once per attempt from the full context. The summaries are a cache rebuilt from the log; the summarizer's calls are not Pi generations: they run as Pi tasks and their usage is in `stats().compactor.usage`, not `pi.usage` (see above).
-- No cache breakpoints inside the view (provider-specific stream options are the host's), no subagent or `note` kinds, no `fsync` log or single-writer lock (Pi's storage owns durability), and the compactor prompt names only the four kinds Pi logs. Summaries are read from the documents on every request and stats call, since tasks write them.
-- Cost: the compactor makes about two calls per message, each carrying the view as context. Choose a cheap summarizer model; the browser-agent example uses the chat model only because it has one.
+- Both views use four-line text blocks with a stable prefix. The compactor folds its own `compactBytes` view of preceding messages, independent of the chat's `viewBytes`. Admission builds preceding parents before dispatching later requests whose context would exceed this budget. Pi 1.2's `Models.complete` accepts retention preferences, but `TextContent` exposes no per-block cache marker. Blocks are unmarked; provider-specific cache-hit guarantees are unavailable. There is no separate `fsync` log or single-writer lock; Pi's storage owns durability. Summaries are read from documents on every request and stats call.
+- Cost: the compactor makes about two calls per message, each carrying its small view as context. Choose a cheap summarizer model; the browser-agent example uses the chat model only because it has one.
 - The package runs in Node and in a browser (no Node-only imports). Fictional-data tests are in `test/packages/agent.test.mjs`; `examples/bot` and `examples/browser-agent` have journeys.
 
+### Forget a message
+
+`await memory.forgetMessage(conversationId, messageIndex)` forgets one message in Pi's current model context. `messageIndex` is its zero-based position in `(await conversation.context(context)).messages`, including system-message positions. System messages cannot be forgotten. This is distinct from the leaf id used by `zoom`; seeded notes and assistant tool calls can shift leaf ids.
+
+The operation replaces every leaf of that message with `(forgotten)` at the same positions. One native commit stores the tombstone and deletes those leaf summaries, their ancestors, later summaries and corresponding failure records. Later summaries can contain facts resolved from earlier context, so deleting only the logarithmic structural ancestors would leave copies behind. Rebuilding can therefore cost more than logarithmic work. Later leaves keep their ids and fingerprints. Native tasks rebuild the invalidated summaries. In-flight calls recheck the memory revision inside their commit and cannot restore old output. A chat request waiting on summaries also rechecks that revision before returning its rewritten messages. `nap` waits for rebuilding. Repeated forgetting is harmless. `forget(id)` only evicts the process cache and does not undo tombstones.
+
+This changes active memory only, not requests already dispatched. Pi's session entries, native task history, underlying storage pages, provider logs, backups and copies quoted in other messages or existing forks are not erased. Indices address the current context, so a reset or history replacement requires the host to resolve positions again. Hosts must authorize the operation. Seed notes are stored once in `boring.memory.optchat.notes`; changing `options.notes` does not overwrite an existing conversation's notes.
+
+Prompt wording and algorithm adaptations retain the upstream Apache-2.0 attribution in [NOTICE](NOTICE) and [the license copy](licenses/Apache-2.0.txt). The file headers identify the changes.
+
 ## Per-request prompt context
 
 `@hachej/boring-agent/prompt-context` (`createPromptContext({ context })`) is one ordinary native extension whose `GenerationHooks.beforeRequest` (registered with `hook(GenerationTask, ...)`) runs before every request attempt, retries and crash recovery included. The hook's text is used for that request only: it is recomputed each time and never appended to the transcript. The text is one user-role `<request-context>...</request-context>` message placed immediately before the round's question when the final message is the user's question, otherwise at the tail (after tool results or an assistant continuation); system messages and earlier messages are unchanged, so the stable prefix stays cacheable.
@@ -270,6 +305,8 @@ Each evidence entry must link to a completed native ToolTask, its original assis
 
 `createSkillsExtension` tells the model that a user message starting with `/<skill-name>` is an explicit request to load that skill first, which is what the chat's `/` menu inserts. `defineAgent(...).skills` lists each skill's name and description for such a menu.
 
+With self-evolution, workspace skills may be flat `.agent/skills/<name>.md` files or `.agent/skills/<name>/SKILL.md` folders. Only `SKILL.md` defines a skill inside a folder; other Markdown and deeper files are companions, not skills. `load_skill` returns the body followed by “This skill's folder is `.agent/skills/<name>`; paths in it are relative to that folder.” (For flat files the folder is `.agent/skills`.) A skill can say `see portions.md` or `see tables/2026.csv`; the agent reads that path under the returned folder with Pi's native `read`, through the same workspace provider. Scanning and loading do not read the companions. No special companion folder, new file tool or additional access is granted. Host-given `Skill` data stays `{ name, description, body }`: with no source folder, `load_skill` returns its body unchanged.
+
 ## Sub-agents and schedules
 
 These follow [Pi Durable on Cloudflare](https://earendil.com/posts/pi-durable-on-cloudflare/) and stay host-agnostic: a Node process and a Durable Object mount the same tools.
@@ -288,7 +325,7 @@ These follow [Pi Durable on Cloudflare](https://earendil.com/posts/pi-durable-on
 
 `defineAgent({ …, selfEvolving: true, workspace })` lets the agent keep its own standing instructions, skills and tools in the `.agent/` folder of its workspace and apply them with a `reload` tool; the plan and its reasons are in [SELF-EVOLUTION.md](../../docs/architecture/SELF-EVOLUTION.md). Off by default.
 
-- One native extension per workspace, named `self-evolving:<workspace>`, so two workspaces in one host never replace each other's tools. Without a fixed `workspace` the agent works in a workspace per conversation: `inWorkspace(name)` returns the agent of workspace `name` (made once, then the same), whose configuration selects only `self-evolving:<name>`; the host creates each conversation with the agent of its owner's workspace and installs and reloads that workspace's extension when the workspace opens, so one harness and one registry never offer one workspace's agent-written tools to another's. It carries the skills (the host's, then `.agent/skills/*.md` parsed with `parseSkill`, behind the usual `load_skill`), `reload`, the agent-written tools and the `agent-written` prompt section. It is selected last; the host's `instructions` become the first section (`host-instructions`) of `agent.<id>`, because Pi renders native `instructions` after every extension section.
+- One native extension per workspace, named `self-evolving:<workspace>`, so two workspaces in one host never replace each other's tools. Without a fixed `workspace` the agent works in a workspace per conversation: `inWorkspace(name)` returns the agent of workspace `name` (made once, then the same), whose configuration selects only `self-evolving:<name>`; the host creates each conversation with the agent of its owner's workspace and installs and reloads that workspace's extension when the workspace opens, so one harness and one registry never offer one workspace's agent-written tools to another's. It carries the skills (the host's, then `.agent/skills/*.md` and `.agent/skills/<name>/SKILL.md` parsed with `parseSkill`, behind the usual `load_skill`), `reload`, the agent-written tools and the `agent-written` prompt section. It is selected last; the host's `instructions` become the first section (`host-instructions`) of `agent.<id>`, because Pi renders native `instructions` after every extension section.
 - `.agent/AGENTS.md` is rendered by that section on every request through the request's environment, labelled as agent-written and capped at `MAX_AGENT_INSTRUCTIONS` (16,000) characters.
 - `.agent/tools/<name>.json` is `{ name, description, parameters, run }`: `name` snake_case and not one of the agent's other tools, `parameters` a JSON Schema with `"type": "object"`, `run` one command line; a path under `.agent/` that `run` names must exist. Each becomes a native `ToolRegistration` (through `jsonSchemaTool`, so Pi refuses arguments that do not match `parameters` before `run` starts) whose `execute` runs `run` through the call's own `ExecutionEnv.exec` (the same provider instance as the files it was read from), with the arguments as one line of JSON on standard input (a quoted here-document); the output is the result and a non-zero exit is an error.
 - `reload` (and `DefinedAgent.reload(env, context)`, which a host calls on open and for the person's `/reload`) reads `.agent/` through the given environment's `FileSystem`, rebuilds the extension and calls the native `registry.install()` in every registry `install` was given (same name: replaced in place). Reloads are serialised. The report (`SelfEvolutionReport`, its `text` is the tool result) names tools added, changed and removed, the agent's skills, the instructions and every load error (invalid JSON, schema, missing script, name conflict, unreadable file). A reload never imports or evaluates anything. Without `exec` the tools are reported unavailable and skipped; instructions and skills still load.

package/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,240 @@
+# Third-party notices
+
+The optional `@hachej/boring-agent/documents` readers load the following dependencies
+only when a document reader runs. Their upstream licenses are reproduced below.
+
+## unpdf 1.9.1 — MIT
+
+Source: https://github.com/unjs/unpdf/tree/v1.9.1
+
+```text
+MIT License
+
+Copyright (c) 2023-PRESENT Johann Schopplich
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+```
+
+## SheetJS xlsx 0.20.3 — Apache-2.0
+
+Distribution: https://cdn.sheetjs.com/xlsx-0.20.3/xlsx-0.20.3.tgz
+
+```text
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "{}"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright (C) 2012-present   SheetJS LLC
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+```

package/dist/documents.d.ts

@@ -0,0 +1,82 @@
+import type { ToolExecutionResult } from '@earendil-works/pi-durable';
+import { Type } from '@earendil-works/pi-ai';
+/** The media type of these bytes; the name only distinguishes CSV from other UTF-8 text. */
+export declare function sniff(bytes: Uint8Array, name?: string): string;
+type Refusal = {
+    path: string;
+    type: string;
+    size: number;
+    error: string;
+};
+type StoredDocument = {
+    path: string;
+    content: string;
+} | {
+    path: string;
+    type: string;
+    size: number;
+} | Refusal;
+type PdfPage = {
+    n: number;
+    text: string;
+    ocr?: true;
+    truncated?: true;
+};
+type PdfResult = Refusal | {
+    path: string;
+    error: string;
+    totalPages: number;
+} | {
+    path: string;
+    totalPages: number;
+    pages: PdfPage[];
+    note?: string;
+    next?: string;
+    untrusted: true;
+};
+type SheetResult = {
+    name: string;
+    rows: number;
+    columns: number;
+    shown: number;
+    csv: string;
+    truncated?: true;
+};
+type SpreadsheetResult = Refusal | {
+    path: string;
+    error: string;
+    sheetNames: string[];
+} | {
+    path: string;
+    sheetNames: string[];
+    moreSheets?: number;
+    sheets: SheetResult[];
+    untrusted: true;
+};
+export type DocumentOcr = (bytes: Uint8Array, pages: readonly number[]) => Promise<{
+    readonly pages?: readonly {
+        readonly n: number;
+        readonly text?: string;
+    }[];
+    readonly provider?: string;
+}>;
+export type DocumentToolsOptions = {
+    readonly readBytes: (path: string) => Uint8Array | Promise<Uint8Array>;
+    readonly ocr?: DocumentOcr;
+    readonly images?: boolean;
+};
+/** Native result for a host's stored-file read path; this does not register or replace Pi's read. */
+export declare function readStored(path: string, bytes: Uint8Array, { images }: {
+    readonly images: boolean;
+}): ToolExecutionResult<StoredDocument>;
+/** Native tool registration is the Pi seam; the host owns byte access, policy and OCR credentials. */
+export declare function documentTools({ readBytes, ocr }: DocumentToolsOptions): (import("@earendil-works/pi-durable").ToolRegistration<Type.TObject<{
+    path: Type.TString;
+    pages: Type.TOptional<Type.TString>;
+}>, PdfResult> | import("@earendil-works/pi-durable").ToolRegistration<Type.TObject<{
+    path: Type.TString;
+    sheet: Type.TOptional<Type.TString>;
+    range: Type.TOptional<Type.TString>;
+}>, SpreadsheetResult>)[];
+export {};
+//# sourceMappingURL=documents.d.ts.map