pi-posthorse 0.10.0 → 0.10.1

package/README.md

@@ -2,11 +2,11 @@
 
 Posthorse starts fresh context windows for long [Pi](https://github.com/earendil-works/pi) tasks without an automatic model-generated summary. Stock Pi already preserves the session journal and retains a recent tail during compaction; Posthorse adds explicit notes and history retrieval, with a handoff or bounded recovery record instead of the automatic summary.
 
-![A Pi conversation rolls into a fresh context window. Task notes and the complete session transcript remain available.](.github/readme/context-rollover.png)
+![A Pi conversation rolls into a fresh context window. Task notes and the complete session transcript remain available.](https://raw.githubusercontent.com/fitchmultz/pi-posthorse/main/.github/readme/context-rollover.png)
 
 ## Install and start
 
-Use Node 24.12.0 or later and Pi 1.3.0 or later. Posthorse supports official Pi and [fitchmultz/pi](https://github.com/fitchmultz/pi).
+Check the [Node.js and Pi requirements](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#install) before installing. Posthorse supports official Pi and [fitchmultz/pi](https://github.com/fitchmultz/pi).
 
 Update an older Pi installation before you install Posthorse. Run these commands:
 
@@ -27,9 +27,13 @@ Posthorse can nudge the model to update task notes before rollover, then continu
 
 Automatic rollover retains a safe recent-message tail using Pi's `compaction.keepRecentTokens`, alongside a bounded recovery record for discarded inputs and tool evidence. It never calls a summarization model. The model must restore notes and check live state before it continues. Optional `new_context` deliberately retains no prior messages; manual `/compact` uses Pi's normal summarization.
 
-Safe recent-tail retention and the notes-only reminder are part of the [0.10.0 changes](CHANGELOG.md#0100). Match these docs to your installed version using the changelog; default-branch documentation may describe changes not yet published.
+Posthorse does not schedule rollover when a provider cache expires. An idle session below the
+rollover threshold keeps its context; use an explicit `new_context` when a deliberate clean reset
+is useful. See [idle sessions and cache expiry](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#idle-sessions-and-cache-expiry).
 
-Recoverable history is not guaranteed model recall. See the [practical guarantees and limits](docs/context-recovery.md) for the stock-Pi comparison, search scope, checkpoints, privacy, concurrency and cost, or the [rollover reference](docs/reference.md#how-it-works) for implementation details.
+See the [changelog](https://github.com/fitchmultz/pi-posthorse/blob/main/CHANGELOG.md) for release history. Default-branch documentation may describe changes not yet published.
+
+Recoverable history is not guaranteed model recall. See the [practical guarantees and limits](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/context-recovery.md) for the stock-Pi comparison, search scope, checkpoints, privacy, concurrency and cost, or the [rollover reference](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#how-it-works) for implementation details.
 
 ## Tools
 
@@ -44,7 +48,7 @@ These tools are available to the model:
 
 Expand a tool card with Ctrl+O. In fullscreen mode, click a tool card to expand it. A `new_context` card shows a request. Pi's `Compacted from … tokens` card confirms the rollover.
 
-See [tool examples and paging details](docs/reference.md#tools).
+See [tool examples and paging details](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#tools).
 
 ## Settings and limits
 
@@ -60,10 +64,10 @@ Two Posthorse copies can register the same tools and compete for rollover. Keep
 
 Multiple compaction-policy extensions can make rollover depend on load order. Enable only one.
 
-Oversized input and some overflow or truncated responses can reach the provider before automatic recovery. Save task state in notes. Background work outside the foreground tool batch continues across rollover.
+Oversized input and some overflow or truncated responses can reach the provider before automatic recovery. Save task state in notes. Posthorse's automatic and explicit rollovers do not stop background work outside the foreground tool batch. Manual `/compact` follows Pi's abort behavior instead.
 
-See [size limits and extension order](docs/reference.md#settings).
-Check the [legacy-session details](docs/reference.md#official-pi-and-the-fork) before you move old fork sessions to official Pi.
+See [size limits and extension order](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#settings).
+Check the [legacy-session details](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#official-pi-and-the-fork) before you move old fork sessions to official Pi.
 
 ## Update or remove
 
@@ -72,7 +76,7 @@ pi update npm:pi-posthorse
 pi remove npm:pi-posthorse
 ```
 
-Removal leaves notes and Pi session history in place. See the [installation reference](docs/reference.md#install) for Git sources.
+Removal leaves notes and Pi session history in place. See the [installation reference](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#install) for Git sources.
 
 ## Data and privacy
 
@@ -80,14 +84,14 @@ Posthorse makes no network requests. Notes are plaintext files under `.pi/notes/
 
 History can include user messages, assistant thinking, tool arguments, errors, and images. Content you retrieve enters the context of your selected model and provider.
 
-See [storage and privacy details](docs/reference.md#data-and-privacy).
-For cost diagnosis, see the [cached-token investigation](ARTIFACTS/investigation-report.md).
+See [storage and privacy details](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#data-and-privacy).
+For cost diagnosis, see the [cached-token investigation](https://github.com/fitchmultz/pi-posthorse/blob/main/ARTIFACTS/investigation-report.md).
 It does not establish the cause of a production bill or a measured cost improvement.
 
 ## Development
 
-See [development and host qualification](docs/development.md) for setup, tests, and releases.
-See [code quality](docs/quality.md) for the required checks.
+See [development and host qualification](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/development.md) for setup, tests, and releases.
+See [code quality](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/quality.md) for the required checks.
 
 ## License
 

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "pi-posthorse",
-	"version": "0.10.0",
+	"version": "0.10.1",
 	"type": "module",
 	"description": "Posthorse: fresh context, same journey. No-summary context rollover for Pi, with sparse budget reminders, handoffs, notes, and history.",
 	"keywords": [
@@ -33,7 +33,7 @@
 	"scripts": {
 		"check": "tsc",
 		"quality:prepare": "node scripts/quality-prepare.ts",
-		"quality:policy": "node scripts/quality-policy.ts",
+		"quality:policy": "node scripts/quality-policy.ts && node scripts/config-policy.ts",
 		"quality:probes": "npm run quality:prepare && node --test test/quality*.test.ts",
 		"lint": "npm run quality:prepare && npm run quality:policy && oxlint --deny-warnings .",
 		"lint:agent": "npm run quality:prepare && npm run quality:policy && oxlint --deny-warnings --format=agent .",
@@ -41,7 +41,7 @@
 		"format": "oxfmt --write .",
 		"format:check": "oxfmt --check .",
 		"build": "npm pack --dry-run",
-		"verify": "npm run lint && npm run format:check && npm run check:compat && npm run build",
+		"verify": "node scripts/toolchain-policy.ts && npm run lint && npm run format:check && npm run check:compat && npm run build",
 		"check:compat": "npm run check && npm test",
 		"test:native": "node --import ./test/pi-loader.ts --test test/host.test.mjs",
 		"test": "npm run quality:prepare && node --import ./test/pi-loader.ts --test test/*.test.ts test/host.test.mjs",
@@ -73,6 +73,7 @@
 		"oxlint": "1.87.0",
 		"oxlint-tsgolint": "7.0.2003",
 		"typebox": "1.3.27",
-		"typescript": "7.0.2"
+		"typescript": "7.0.2",
+		"yaml": "2.9.1"
 	}
 }

package/src/budget.ts

@@ -137,10 +137,10 @@ export function freshPayloadChars(
 		) * 4,
 	);
 }
-export function pageCapacity(ctx: PolicyContext, toolTokens: number): number {
+export function pageCapacity(ctx: PolicyContext, toolTokens: () => number): number {
 	const usage = ctx.getContextUsage();
 	if (usage === undefined || usage.tokens === null) {
-		return freshPayloadChars(ctx, toolTokens, [], MAX_PAGE_CHARS) / 4 + PAGE_MARGIN_TOKENS;
+		return freshPayloadChars(ctx, toolTokens(), [], MAX_PAGE_CHARS) / 4 + PAGE_MARGIN_TOKENS;
 	}
 	const budget = budgetFor(ctx, usage.contextWindow);
 	const line =

package/src/files.ts

@@ -21,6 +21,7 @@ import {
 	rename,
 	stat,
 	unlink,
+	type FileHandle,
 } from "node:fs/promises";
 import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
 import { getAgentDir } from "@earendil-works/pi-coding-agent";
@@ -123,31 +124,50 @@ export async function publishFile(
 		throw error;
 	}
 }
-/** Raw LF-delimited lines; UTF-8 never encodes LF inside another character. */
-export async function* lfLines(
+/** Bounded LF-delimited segments; end also closes an unterminated final line at EOF. */
+export async function* lfChunks(
 	file: string,
 	signal?: AbortSignal,
 	highWaterMark = 64 * 1024,
-): AsyncGenerator<Buffer> {
-	const stream = createReadStream(file, { signal, highWaterMark });
-	let pending: Buffer[] = [];
+	handle?: Readonly<Pick<FileHandle, "createReadStream">>,
+): AsyncGenerator<{ readonly bytes: Buffer; readonly end: boolean }> {
+	const stream =
+		handle === undefined
+			? createReadStream(file, { signal, highWaterMark })
+			: handle.createReadStream({ signal, highWaterMark, autoClose: false });
+	let openLine = false;
 	for await (const chunk of stream) {
 		if (!Buffer.isBuffer(chunk)) {
 			throw new Error("Expected a byte stream.");
 		}
 		let start = 0;
 		for (let end = chunk.indexOf(10); end !== -1; end = chunk.indexOf(10, start)) {
-			const line = chunk.subarray(start, end);
-			yield pending.length === 0 ? line : Buffer.concat([...pending, line]);
-			pending = [];
+			yield { bytes: chunk.subarray(start, end), end: true };
+			openLine = false;
 			start = end + 1;
 		}
 		if (start < chunk.length) {
-			pending.push(chunk.subarray(start));
+			yield { bytes: chunk.subarray(start), end: false };
+			openLine = true;
 		}
 	}
-	if (pending.length > 0) {
-		yield Buffer.concat(pending);
+	if (openLine) {
+		yield { bytes: Buffer.alloc(0), end: true };
+	}
+}
+/** Raw LF-delimited lines; UTF-8 never encodes LF inside another character. */
+export async function* lfLines(
+	file: string,
+	signal?: AbortSignal,
+	highWaterMark = 64 * 1024,
+): AsyncGenerator<Buffer> {
+	let pending: Buffer[] = [];
+	for await (const { bytes, end } of lfChunks(file, signal, highWaterMark)) {
+		pending.push(bytes);
+		if (end) {
+			yield pending.length === 1 ? bytes : Buffer.concat(pending);
+			pending = [];
+		}
 	}
 }
 /** Git's heuristic: a NUL in the first 8,000 bytes marks binary content. */

package/src/history-search.ts

@@ -1,4 +1,5 @@
 import { relative, resolve } from "node:path";
+import type { LineMatcher } from "./session-lines.ts";
 import {
 	isReminderType,
 	requireValue,
@@ -12,7 +13,6 @@ import { shortKey } from "./message-text.ts";
 import {
 	needleLines,
 	prefilterNeedles,
-	type RawLine,
 	scopedSessionFiles,
 	sessionWindowEntries,
 } from "./session-files.ts";
@@ -82,21 +82,30 @@ function entryMessage(entry: EntryLike) {
 			return { content: entry.content, summary: entry.summary };
 	}
 }
-function contextVisibility(projected: readonly ProjectedEntry[]): (entry: EntryLike) => boolean {
-	const visible = new Map(
-		projected.flatMap(({ sourceEntry, messages }) =>
-			sourceEntry.id !== undefined && sourceEntry.id !== "" && messages.length > 0
-				? [[sourceEntry.id, messages.map(seenText).join("\n")]]
-				: [],
-		),
-	);
+function contextVisibility(project: HistorySource["projection"]): (entry: EntryLike) => boolean {
+	let visible: Map<string, ProjectedEntry["messages"] | string> | undefined;
 	return (entry) => {
 		if (entry.type === "custom_message" && isReminderType(entry.customType)) {
 			return false;
 		}
-		const own = entryMessage(entry);
-		const id = entry.type === "context_edit" ? entry.targetId : entry.id;
-		return visible.get(id ?? "")?.includes(seenText(own)) === true;
+		// Pi owns edits and compaction; only matching candidates need their projected text.
+		visible ??= new Map(
+			project().flatMap(({ sourceEntry, messages }) =>
+				sourceEntry.id !== undefined && sourceEntry.id !== "" && messages.length > 0
+					? [[sourceEntry.id, messages]]
+					: [],
+			),
+		);
+		const id = (entry.type === "context_edit" ? entry.targetId : entry.id) ?? "";
+		let text = visible.get(id);
+		if (text === undefined) {
+			return false;
+		}
+		if (typeof text !== "string") {
+			text = text.map(seenText).join("\n");
+			visible.set(id, text);
+		}
+		return text.includes(seenText(entryMessage(entry)));
 	};
 }
 type SearchProgress = {
@@ -193,14 +202,19 @@ async function scanArchive(
 		readonly source: string;
 		readonly query: string;
 		readonly signal?: AbortSignal;
-		readonly wanted?: (bytes: RawLine) => boolean;
+		readonly wanted?: LineMatcher;
 		readonly inContext: (entry: EntryLike) => boolean;
 	},
 	accumulator: SearchProgress,
 ): Promise<{ readonly groups: readonly (readonly HistoryHit[])[]; readonly skipped: number }> {
 	const collector = archiveCollector(accumulator);
 	let skipped = 0;
-	for await (const item of sessionWindowEntries(file, options.signal, options.wanted)) {
+	for await (const item of sessionWindowEntries(
+		file,
+		options.signal,
+		undefined,
+		options.wanted,
+	)) {
 		const hit = historyHit(item, options.query, options.source);
 		if (hit === undefined) {
 			continue;
@@ -299,7 +313,9 @@ export async function historyMatches(
 		fail("Invalid history cursor or changed query/scope; restart the search without it.");
 	}
 	const accumulator = new SearchAccumulator(limit, cursor, fail);
-	const inContext = contextVisibility(source.projection());
+	// Archive scans await I/O; keep their visibility snapshot at invocation time.
+	const projected = params.all === true ? source.projection() : undefined;
+	const inContext = contextVisibility(() => projected ?? source.projection());
 	const skipped =
 		params.all === true
 			? await archivedMatches(source, { query, signal, inContext }, accumulator)

package/src/history.ts

@@ -50,7 +50,7 @@ async function findEntry(
 		for await (const item of sessionWindowEntries(
 			file,
 			signal,
-			(_bytes, entry) => entry.id === entryId,
+			(entry) => entry.id === entryId,
 		)) {
 			return { item, source: relative(source.dir, file) };
 		}

package/src/notes.ts

@@ -1,13 +1,7 @@
-import {
-	appendFileSync,
-	existsSync,
-	mkdirSync,
-	readdirSync,
-	readFileSync,
-	realpathSync,
-	statSync,
-} from "node:fs";
+import { appendFileSync, existsSync, mkdirSync, readFileSync, statSync } from "node:fs";
+import { readdir, realpath, stat } from "node:fs/promises";
 import { dirname, isAbsolute, join, normalize, sep } from "node:path";
+import { setImmediate } from "node:timers/promises";
 import {
 	formatSize,
 	withFileMutationQueue,
@@ -18,6 +12,7 @@ import { Type } from "typebox";
 import { toolCards } from "../ui.ts";
 import {
 	MAX_PAGE_CHARS,
+	isRecord,
 	requireValue,
 	textResult,
 	type PolicyContext,
@@ -36,10 +31,25 @@ export type NotesParams = {
 };
 type NoteFile = { readonly path: string; readonly size: number; readonly mtimeMs: number };
 type NoteHits = { readonly rows: string[]; readonly more: boolean; readonly skipped: number };
+/** Paged operations take a fresh policy snapshot after their filesystem awaits. */
+type NotesAccess = {
+	readonly host: () => PolicyContext;
+	readonly pages: PageAccess;
+	readonly signal: AbortSignal | undefined;
+};
 /** Larger files are logs or evidence rather than notes; reading them would dominate a search. */
 const SEARCH_FILE_BYTES = 1024 * 1024;
 /** Overlapping reads cut a full search of a 212,000-file notes tree from 39 s to 14 s. */
 const SEARCH_BATCH = 16;
+/**
+ * Maximum explicit `stat` calls overlapped per batch. On a 212,751-note tree, 16, 64 and 256 took
+ * equal time within host noise. The walk enters one folder at a time, so this bounds our pending
+ * `stat` calls at any depth. On filesystems that report unknown entry types, Node 24's
+ * `readdir({ withFileTypes: true })` also issues its own `lstat` for every entry at once.
+ */
+const WALK_BATCH = 64;
+/** Formatting all 212,751 rows of one listing at once stalled Pi for 156-163 ms. */
+const FORMAT_BATCH = 10_000;
 /** Search rescans up to the requested offset, so this also caps the matches one call keeps. */
 const SEARCH_OFFSET_LIMIT = 1_000_000;
 function safeJoin(dir: string, path: string): string {
@@ -49,46 +59,105 @@ function safeJoin(dir: string, path: string): string {
 	}
 	return join(dir, relative);
 }
-function walk(directory: string, ancestors: readonly string[] = []): NoteFile[] {
-	const root = realpathSync(directory);
+/** Like `statSync(path, { throwIfNoEntry: false })`: an entry deleted mid-walk is skipped. */
+async function statIfPresent(path: string) {
+	try {
+		return await stat(path);
+	} catch (error) {
+		if (isRecord(error) && (error.code === "ENOENT" || error.code === "ENOTDIR")) {
+			return;
+		}
+		throw error;
+	}
+}
+/** A folder's present entries in listing order. */
+async function* folderEntries(directory: string, signal: AbortSignal | undefined) {
+	const entries = await readdir(directory, { withFileTypes: true });
+	let start = 0;
+	while (start < entries.length) {
+		signal?.throwIfAborted();
+		const batch = entries.slice(start, start + WALK_BATCH);
+		const firstFolder = batch.findIndex(
+			(entry) => entry.isDirectory() || entry.isSymbolicLink(),
+		);
+		const paths = batch
+			.slice(0, firstFolder === -1 ? batch.length : firstFolder + 1)
+			.map((entry) => join(directory, entry.name));
+		// Stats overlap only up to the next entry that may be a folder.
+		// oxlint-disable-next-line no-await-in-loop
+		const infos = await Promise.all(paths.map(statIfPresent));
+		for (const [index, path] of paths.entries()) {
+			start += 1;
+			const info = infos[index];
+			if (info !== undefined) {
+				yield { path, info };
+			}
+			if (info?.isDirectory() === true) {
+				// Stat later entries only after this folder's subtree is walked, discarding any
+				// already taken, so entries removed meanwhile are skipped.
+				break;
+			}
+		}
+	}
+}
+/** Files depth-first in directory-listing order, yielding between filesystem calls. */
+async function walk(
+	directory: string,
+	signal: AbortSignal | undefined,
+	ancestors: readonly string[] = [],
+): Promise<NoteFile[]> {
+	signal?.throwIfAborted();
+	const root = await realpath(directory);
 	if (ancestors.includes(root)) {
 		return [];
 	}
 	const next = [...ancestors, root];
-	return readdirSync(directory).flatMap((file) => {
-		const path = join(directory, file);
-		const info = statSync(path, { throwIfNoEntry: false });
-		if (info === undefined) {
-			return [];
+	const files: NoteFile[] = [];
+	for await (const { path, info } of folderEntries(directory, signal)) {
+		if (info.isDirectory()) {
+			for (const file of await walk(path, signal, next)) {
+				files.push(file);
+			}
+		} else {
+			files.push({ path, size: info.size, mtimeMs: info.mtimeMs });
 		}
-		return info.isDirectory()
-			? walk(path, next)
-			: [{ path, size: info.size, mtimeMs: info.mtimeMs }];
-	});
+	}
+	return files;
 }
-function notesUnder(target: string): NoteFile[] {
-	const info = statSync(target, { throwIfNoEntry: false });
+async function notesUnder(target: string, signal: AbortSignal | undefined): Promise<NoteFile[]> {
+	const info = await statIfPresent(target);
+	signal?.throwIfAborted();
 	if (info === undefined) {
 		return [];
 	}
 	const files = info.isDirectory()
-		? walk(target)
+		? await walk(target, signal)
 		: [{ path: target, size: info.size, mtimeMs: info.mtimeMs }];
+	signal?.throwIfAborted();
 	return files.sort((a, b) => b.mtimeMs - a.mtimeMs);
 }
-function listNotes(
+async function listNotes(
 	dir: string,
 	params: NotesParams,
-	host: PolicyContext,
-	pages: PageAccess,
-): TextResult {
+	access: NotesAccess,
+): Promise<TextResult> {
+	const { host, pages, signal } = access;
 	const folder = params.path;
 	const target = folder === undefined || folder === "" ? dir : safeJoin(dir, folder);
-	const rows = notesUnder(target).map(
-		(file) =>
-			`${file.path.slice(dir.length + 1)}  ${formatSize(file.size)}  ${new Date(file.mtimeMs).toISOString().slice(0, 16)}Z`,
-	);
-	return pages.notes(host, {
+	const files = await notesUnder(target, signal);
+	const rows: string[] = [];
+	for (let start = 0; start < files.length; start += FORMAT_BATCH) {
+		// Pi gets an event-loop turn before each batch of rows.
+		// oxlint-disable-next-line no-await-in-loop
+		await setImmediate();
+		signal?.throwIfAborted();
+		for (const file of files.slice(start, start + FORMAT_BATCH)) {
+			rows.push(
+				`${file.path.slice(dir.length + 1)}  ${formatSize(file.size)}  ${new Date(file.mtimeMs).toISOString().slice(0, 16)}Z`,
+			);
+		}
+	}
+	return pages.notes(host(), {
 		rows,
 		offset: params.offset ?? 0,
 		kind: "notes-list",
@@ -99,9 +168,9 @@ function listNotes(
 async function readNote(
 	dir: string,
 	params: NotesParams,
-	getHost: () => PolicyContext,
-	pages: PageAccess,
+	access: NotesAccess,
 ): Promise<TextResult> {
+	const { pages } = access;
 	const relative = requireValue(params.path, "path", params.op);
 	const path = safeJoin(dir, relative);
 	const info = statSync(path, { throwIfNoEntry: false });
@@ -109,7 +178,7 @@ async function readNote(
 		throw new Error(`No note at ${relative}. Use op "list" to see available notes.`);
 	}
 	if (info.isDirectory()) {
-		return listNotes(dir, params, getHost(), pages);
+		return listNotes(dir, params, access);
 	}
 	if (await isBinaryFile(path)) {
 		throw new Error(
@@ -122,7 +191,7 @@ async function readNote(
 		throw new Error(`Offset ${offset} is past the end of ${relative} (${text.length} chars).`);
 	}
 	const header = `File: ${path}\n`;
-	const host = getHost();
+	const host = access.host();
 	const chars = pages.size(host, { offset }) - header.length;
 	if (chars <= 0) {
 		pages.error(
@@ -197,11 +266,13 @@ async function noteHits(
 	limit: number,
 	signal?: AbortSignal,
 ): Promise<NoteHits> {
-	const notes = notesUnder(dir);
+	const notes = await notesUnder(dir, signal);
 	const rows: string[] = [];
 	let length = -1;
 	let skipped = 0;
 	for (let start = 0; start < notes.length; start += SEARCH_BATCH) {
+		// A batch of notes over 1 MiB reads nothing that would notice cancellation.
+		signal?.throwIfAborted();
 		// Batches keep note order, so a search stops within one batch of the page limit.
 		// oxlint-disable-next-line no-await-in-loop
 		const batch = await Promise.all(
@@ -231,21 +302,20 @@ async function noteHits(
 async function searchNotes(
 	dir: string,
 	params: NotesParams,
-	access: { readonly host: PolicyContext; readonly pages: PageAccess },
-	signal?: AbortSignal,
+	access: NotesAccess,
 ): Promise<TextResult> {
 	const { host, pages } = access;
 	const query = requireValue(params.query, "query", params.op);
 	const offset = params.offset ?? 0;
 	if (offset >= SEARCH_OFFSET_LIMIT) {
 		pages.error(
-			host,
+			host(),
 			`Notes search pages end before offset ${SEARCH_OFFSET_LIMIT}; search for something more specific.`,
 		);
 	}
-	const hits = await noteHits(dir, query.toLowerCase(), offset + MAX_PAGE_CHARS, signal);
+	const hits = await noteHits(dir, query.toLowerCase(), offset + MAX_PAGE_CHARS, access.signal);
 	const skipped = hits.skipped === 1 ? "1 note" : `${hits.skipped} notes`;
-	return pages.notes(host, {
+	return pages.notes(host(), {
 		rows: hits.rows,
 		more: hits.more,
 		offset,
@@ -297,17 +367,18 @@ export function registerNotes(
 		// oxlint-disable-next-line max-params
 		async execute(_id, params, signal, _onUpdate, ctx) {
 			const dir = notesDir(ctx.cwd);
+			const access: NotesAccess = { host: () => policy(ctx), pages, signal };
 			switch (params.op) {
 				case "list":
-					return listNotes(dir, params, policy(ctx), pages);
+					return listNotes(dir, params, access);
 				case "read":
-					return readNote(dir, params, () => policy(ctx), pages);
+					return readNote(dir, params, access);
 				case "write":
 					return writeNote(dir, params, signal);
 				case "append":
 					return appendNote(dir, params, signal);
 				case "search":
-					return searchNotes(dir, params, { host: policy(ctx), pages }, signal);
+					return searchNotes(dir, params, access);
 			}
 		},
 	});

package/src/paging.ts

@@ -63,7 +63,7 @@ export class PageReservations {
 			);
 		}
 		this.previousUsage = usage;
-		const remaining = pageCapacity(ctx, this.toolTokens()) - this.pendingTokens;
+		const remaining = pageCapacity(ctx, this.toolTokens) - this.pendingTokens;
 		const chars = Math.min(
 			MAX_PAGE_CHARS,
 			Math.max(0, remaining - PAGE_MARGIN_TOKENS) * 4 -

package/src/recovery-results.ts

@@ -9,7 +9,7 @@ import {
 	toolIdentity,
 } from "./message-text.ts";
 
-export type RecoveryEntry = EntryLike & { readonly sourceEntryId: string | undefined };
+export type RecoveryEntry = EntryLike & { readonly retained: boolean };
 type LinkedCall = { readonly entry: EntryLike; readonly block: ToolCallLike };
 export type RecoveryBlock = { readonly header: string; readonly text: string };
 export type RecoveryBatch = { readonly callId?: string; readonly blocks: readonly RecoveryBlock[] };
@@ -85,17 +85,12 @@ function resultBlock(
 /** Async receipts may arrive after a response; completion does not establish receipt delivery. */
 export function recoverableToolResults(
 	entries: readonly RecoveryEntry[],
-	retained: ReadonlySet<string>,
 ): RecoveryBatch | undefined {
 	const calls = callsById(entries);
 	const start = trailingStart(entries);
 	const results = entries.filter((entry, index) => {
 		const message = entry.message;
-		if (
-			message?.role !== "toolResult" ||
-			message.toolCallId === undefined ||
-			retained.has(entry.sourceEntryId ?? "")
-		) {
+		if (message?.role !== "toolResult" || message.toolCallId === undefined || entry.retained) {
 			return false;
 		}
 		const call = calls.get(message.toolCallId);

package/src/recovery.ts

@@ -265,6 +265,7 @@ function skippedRecords(ids: readonly string[], limit: number): string | undefin
 function projectedToolEntries(
 	entries: readonly EntryLike[],
 	projected: readonly ProjectedEntry[],
+	retained: ReadonlySet<string>,
 ): RecoveryEntry[] {
 	const edits = contextEdits(entries);
 	return projected.flatMap(({ sourceEntry, messages }) =>
@@ -272,8 +273,8 @@ function projectedToolEntries(
 			.filter((message) => message.role === "assistant" || message.role === "toolResult")
 			.map((message) => ({
 				type: "message",
-				// Retention follows the source; history reads follow the effective replacement.
-				sourceEntryId: sourceEntry.id,
+				// Resolve retention before replacing identity with the history-read reference.
+				retained: retained.has(sourceEntry.id ?? ""),
 				id: edits.get(sourceEntry.id ?? "")?.id ?? sourceEntry.id,
 				message,
 			})),
@@ -442,7 +443,7 @@ export function buildAutoHandoff(
 		fixed.length > 0
 			? "Selected owner and coordination inputs (chronological):"
 			: emptyInputHeader;
-	const batch = recoverableToolResults(projectedToolEntries(entries, projected), retained);
+	const batch = recoverableToolResults(projectedToolEntries(entries, projected, retained));
 	const header = batchHeader(batch);
 	const minimum = [
 		preamble,

package/src/session-files.ts

@@ -12,6 +12,7 @@ import {
 	type WindowedEntry,
 } from "./contracts.ts";
 import { lfLines } from "./files.ts";
+import { sessionLines, type LineMatcher, type SessionLine } from "./session-lines.ts";
 import { toWindowedEntry, windowProjection } from "./history-entries.ts";
 import { shortKey } from "./message-text.ts";
 
@@ -78,11 +79,11 @@ const ASCII_FOLDS: Readonly<Partial<Record<string, string>>> = {
 	k: "\u00e2\u0084\u00aa",
 	s: "\u00c5\u00bf",
 };
-const DOTTED_I = Buffer.from("\u0130");
-/** The read capabilities line filters need from a raw journal line. */
-export type RawLine = Readonly<Pick<Buffer, "includes" | "toString">>;
-/** Whether a raw line contains one of prefilterNeedles' needles, ignoring case like its text would. */
-export function needleLines(needles: readonly string[]): (bytes: RawLine) => boolean {
+const DOTTED_I = Buffer.from("\u0130").toString("latin1");
+/** Raw-line matcher for prefilterNeedles, ignoring case like the rendered text would. */
+export function needleLines(needles: readonly string[]): LineMatcher {
+	const overlap =
+		4 * needles.reduce((longest, needle) => Math.max(longest, needle.length), 0) + 4;
 	const others = needles.filter((needle) => needle !== "\u0130");
 	const dotted = others.length < needles.length;
 	if (others.every((needle) => /^[\0-\x7f]*$/u.test(needle))) {
@@ -97,11 +98,14 @@ export function needleLines(needles: readonly string[]): (bytes: RawLine) => boo
 				.join(""),
 		);
 		const pattern = new RegExp(latin1.join("|"), "i");
-		return (bytes) =>
-			(dotted && bytes.includes(DOTTED_I)) || pattern.test(bytes.toString("latin1"));
+		return {
+			test: (text) => (dotted && text.includes(DOTTED_I)) || pattern.test(text),
+			encoding: "latin1",
+			overlap,
+		};
 	}
 	const pattern = new RegExp(needles.map((needle) => RegExp.escape(needle)).join("|"), "iu");
-	return (bytes) => pattern.test(bytes.toString("utf8"));
+	return { test: (text) => pattern.test(text), encoding: "utf8", overlap };
 }
 /** File lists that each fit one ripgrep command line. */
 function commandGroups(paths: readonly string[]): string[][] {
@@ -210,8 +214,8 @@ function parsedEntry(line: string): EntryLike | undefined {
 	} // Pi skips malformed journal lines too.
 }
 /** Window metadata of a Pi-ordered line that cannot open a window, without parsing it. */
-function entryHead(bytes: RawLine): EntryLike | undefined {
-	const head = ENTRY_HEAD.exec(bytes.toString("utf8", 0, 512));
+function entryHead(prefix: string): EntryLike | undefined {
+	const head = ENTRY_HEAD.exec(prefix);
 	if (head === null || head[1] === "compaction" || head[1] === "context_window") {
 		return undefined;
 	}
@@ -222,31 +226,40 @@ function entryHead(bytes: RawLine): EntryLike | undefined {
 	};
 }
 /** Whether a caller may need a line; other lines only advance window ancestry. */
-type LineFilter = (bytes: RawLine, entry: EntryLike) => boolean;
+type LineFilter = (entry: EntryLike) => boolean;
+async function readEntry(
+	line: SessionLine,
+	wanted: LineFilter,
+): Promise<{ readonly entry: EntryLike; readonly selected: boolean } | undefined> {
+	const head = entryHead(line.head);
+	if (head !== undefined && (!line.matches || !wanted(head))) {
+		return { entry: head, selected: false };
+	}
+	const entry = parsedEntry(await line.text());
+	return entry === undefined
+		? undefined
+		: {
+				entry,
+				selected: head !== undefined || (line.matches && wanted(entry)),
+			};
+}
 export async function* sessionWindowEntries(
 	file: string,
 	signal?: AbortSignal,
 	wanted: LineFilter = () => true,
+	matcher?: LineMatcher,
 ): AsyncGenerator<WindowedEntry> {
 	if (!existsSync(file)) {
 		return;
 	}
 	const project = windowProjection();
-	for await (const bytes of lfLines(file, signal, 1024 * 1024)) {
-		const head = entryHead(bytes);
-		if (head !== undefined && !wanted(bytes, head)) {
-			project(head);
+	for await (const line of sessionLines(file, signal, matcher)) {
+		const candidate = await readEntry(line, wanted);
+		if (candidate === undefined) {
 			continue;
 		}
-		const entry = parsedEntry(bytes.toString("utf8"));
-		if (entry === undefined) {
-			continue;
-		}
-		const windowId = project(entry);
-		const item =
-			head !== undefined || wanted(bytes, entry)
-				? toWindowedEntry(entry, windowId)
-				: undefined;
+		const windowId = project(candidate.entry);
+		const item = candidate.selected ? toWindowedEntry(candidate.entry, windowId) : undefined;
 		if (item !== undefined) {
 			yield item;
 		}

package/src/session-lines.ts

@@ -0,0 +1,108 @@
+import { open, type FileHandle } from "node:fs/promises";
+import { StringDecoder } from "node:string_decoder";
+import { lfChunks } from "./files.ts";
+
+export type LineMatcher = {
+	readonly test: (text: string) => boolean;
+	readonly encoding: "latin1" | "utf8";
+	/** Enough characters to include every case-fold alternative of any needle. */
+	readonly overlap: number;
+};
+export type SessionLine = {
+	readonly head: string;
+	readonly matches: boolean;
+	readonly text: () => Promise<string>;
+};
+type LineRange = {
+	readonly start: number;
+	readonly end: number;
+	readonly head: string;
+	readonly matches: boolean;
+	readonly bytes?: Readonly<Pick<Buffer, "toString">>;
+};
+async function* lineRanges(
+	file: string,
+	handle: Readonly<Pick<FileHandle, "createReadStream">>,
+	signal: AbortSignal | undefined,
+	matcher: LineMatcher | undefined,
+): AsyncGenerator<LineRange> {
+	let start = 0;
+	let position = 0;
+	let head = Buffer.alloc(0);
+	let tail = "";
+	let decoder = new StringDecoder(matcher?.encoding);
+	let matches = matcher === undefined;
+	for await (const { bytes, end } of lfChunks(file, signal, 1024 * 1024, handle)) {
+		if (head.length < 512) {
+			head = Buffer.concat([head, bytes.subarray(0, 512 - head.length)]);
+		}
+		if (!matches && matcher !== undefined) {
+			const window = tail + decoder.write(bytes) + (end ? decoder.end() : "");
+			matches = matcher.test(window);
+			tail = window.slice(-matcher.overlap);
+		}
+		position += bytes.length;
+		if (end) {
+			yield {
+				start,
+				end: position,
+				head: head.toString("utf8"),
+				matches,
+				bytes: position - start === bytes.length ? bytes : undefined,
+			};
+			position++;
+			start = position;
+			head = Buffer.alloc(0);
+			tail = "";
+			decoder = new StringDecoder(matcher?.encoding);
+			matches = matcher === undefined;
+		}
+	}
+}
+async function rangeText(
+	handle: Readonly<Pick<FileHandle, "read">>,
+	line: LineRange,
+	signal?: AbortSignal,
+): Promise<string> {
+	signal?.throwIfAborted();
+	if (line.bytes !== undefined) {
+		return line.bytes.toString("utf8");
+	}
+	const buffer = Buffer.alloc(line.end - line.start);
+	let offset = 0;
+	while (offset < buffer.length) {
+		signal?.throwIfAborted();
+		// Positioned reads may be short; each continues the same selected record.
+		// oxlint-disable-next-line no-await-in-loop
+		const { bytesRead } = await handle.read(
+			buffer,
+			offset,
+			buffer.length - offset,
+			line.start + offset,
+		);
+		if (bytesRead === 0) {
+			throw new Error("Session file changed while reading a history entry.");
+		}
+		offset += bytesRead;
+	}
+	return buffer.toString("utf8");
+}
+/** Irrelevant lines retain only their prefix, byte range and needle overlap, not their payload. */
+export async function* sessionLines(
+	file: string,
+	signal?: AbortSignal,
+	matcher?: LineMatcher,
+): AsyncGenerator<SessionLine> {
+	const handle = await open(file, "r");
+	try {
+		for await (const line of lineRanges(file, handle, signal, matcher)) {
+			yield {
+				head: line.head,
+				matches: line.matches,
+				text: () => rangeText(handle, line, signal),
+			};
+		}
+	} finally {
+		await handle.close();
+	}
+}